these 12 derivations will be built: /nix/store/m930ifs0krn7cbs6di050ymqlf2w07bs-unit-nixbot.service.drv /nix/store/x0j5lpd6dacwfxwz0ndkvqdzxk6hlxrs-system-units.drv /nix/store/inyas46sxskf51ydqbf1dixq5mxl8ydz-etc.drv /nix/store/8ygid4pcs1lg9idq2m6h9lbdsqfrs6jk-activate.drv /nix/store/v6aic88wccg2hlis82jvaq34aq81makj-nixos-system-gitlab-test.drv /nix/store/7piim8gmr9nxxcwninyhpijfxk8jj6jb-closure-info.drv /nix/store/sajc29pqpqqwgzqk23m86jimx4s22iv9-closure-info.drv /nix/store/fs92rhi45390n8pb1894vr6wg7kbylsx-run-nixos-vm.drv /nix/store/pvl1h72yjnfmnbfqj50c1n1b1s65dd7c-nixos-vm.drv /nix/store/nq03h9j8c7v8q6534bpnsnk69fhm2hih-driverConfiguration.json.drv /nix/store/51bfizgw9bfg27hfp5d871ysy14lsmz6-nixos-test-driver-nixbot-gitlab.drv /nix/store/xgm22mp1yz4jz4cpa7n7mpkyxifwyj63-vm-test-run-nixbot-gitlab.drv building '/nix/store/m930ifs0krn7cbs6di050ymqlf2w07bs-unit-nixbot.service.drv' building '/nix/store/x0j5lpd6dacwfxwz0ndkvqdzxk6hlxrs-system-units.drv' building '/nix/store/inyas46sxskf51ydqbf1dixq5mxl8ydz-etc.drv' building '/nix/store/8ygid4pcs1lg9idq2m6h9lbdsqfrs6jk-activate.drv' building '/nix/store/v6aic88wccg2hlis82jvaq34aq81makj-nixos-system-gitlab-test.drv' nixos-system-gitlab-test> structuredAttrs is enabled building '/nix/store/7piim8gmr9nxxcwninyhpijfxk8jj6jb-closure-info.drv' closure-info> structuredAttrs is enabled building '/nix/store/sajc29pqpqqwgzqk23m86jimx4s22iv9-closure-info.drv' closure-info> structuredAttrs is enabled building '/nix/store/fs92rhi45390n8pb1894vr6wg7kbylsx-run-nixos-vm.drv' building '/nix/store/pvl1h72yjnfmnbfqj50c1n1b1s65dd7c-nixos-vm.drv' building '/nix/store/nq03h9j8c7v8q6534bpnsnk69fhm2hih-driverConfiguration.json.drv' driverConfiguration.json> structuredAttrs is enabled building '/nix/store/51bfizgw9bfg27hfp5d871ysy14lsmz6-nixos-test-driver-nixbot-gitlab.drv' nixos-test-driver-nixbot-gitlab> Running type check (enable/disable: config.skipTypeCheck) nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck nixos-test-driver-nixbot-gitlab> All checks passed! nixos-test-driver-nixbot-gitlab> Linting test script (enable/disable: config.skipLint) nixos-test-driver-nixbot-gitlab> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint nixos-test-driver-nixbot-gitlab> All checks passed! building '/nix/store/xgm22mp1yz4jz4cpa7n7mpkyxifwyj63-vm-test-run-nixbot-gitlab.drv' on 'ssh-ng://nix@jamie' building '/nix/store/xgm22mp1yz4jz4cpa7n7mpkyxifwyj63-vm-test-run-nixbot-gitlab.drv' vm-test-run-nixbot-gitlab> Machine state will be reset. To keep it, pass --keep-machine-state vm-test-run-nixbot-gitlab> start all VLans vm-test-run-nixbot-gitlab> (finished: start all VLans, in 0.01 seconds) vm-test-run-nixbot-gitlab> Test will time out and terminate in 3600 seconds vm-test-run-nixbot-gitlab> run the VM test script vm-test-run-nixbot-gitlab> additionally exposed symbols: vm-test-run-nixbot-gitlab> gitlab, vm-test-run-nixbot-gitlab> vlan1, vm-test-run-nixbot-gitlab> start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh vm-test-run-nixbot-gitlab> start all VMs vm-test-run-nixbot-gitlab> gitlab: starting vm vm-test-run-nixbot-gitlab> gitlab: QEMU running (pid 13) vm-test-run-nixbot-gitlab> gitlab # Disk image does not exist, creating the virtualisation disk image... vm-test-run-nixbot-gitlab> gitlab # Formatting '/build/vm-state-gitlab/tmp.CrpmOCBG0w', fmt=raw size=1073741824 vm-test-run-nixbot-gitlab> gitlab # mke2fs 1.47.3 (8-Jul-2025) vm-test-run-nixbot-gitlab> gitlab # Discarding device blocks: 0/262144 done vm-test-run-nixbot-gitlab> gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes vm-test-run-nixbot-gitlab> gitlab # Filesystem UUID: 0fcc4280-dcbd-4bd9-bcc9-80c7af117e8e vm-test-run-nixbot-gitlab> gitlab # Superblock backups stored on blocks: vm-test-run-nixbot-gitlab> gitlab # 32768, 98304, 163840, 229376 vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # Allocating group tables: 0/8 done vm-test-run-nixbot-gitlab> gitlab # Writing inode tables: 0/8 done vm-test-run-nixbot-gitlab> gitlab # Creating journal (8192 blocks): done vm-test-run-nixbot-gitlab> gitlab # Writing superblocks and filesystem accounting information: 0/8 done vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # Virtualisation disk image created. vm-test-run-nixbot-gitlab> gitlab # Creating Nix store image... vm-test-run-nixbot-gitlab> gitlab # Created Nix store image. vm-test-run-nixbot-gitlab> (finished: start all VMs, in 34.62 seconds) vm-test-run-nixbot-gitlab> gitlab: waiting for unit setup-gitlab.service vm-test-run-nixbot-gitlab> gitlab: waiting for the VM to finish booting vm-test-run-nixbot-gitlab> gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 vm-test-run-nixbot-gitlab> gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # Booting from ROM... vm-test-run-nixbot-gitlab> gitlab # Probing EDD (edd=off to disable)... ok vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun 1 15:51:08 UTC 2026 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5r5nssc2dlvpnlmpdafhh1xdx2dv0q9n-nixos-system-gitlab-test/init regInfo=/nix/store/zr2rs44m0sag591c92993637bihz9h43-closure-info/registration console=ttyS0,115200n8 console=tty0 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-provided physical RAM map: vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] NX (Execute Disable) protection: active vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] APIC: Static calls initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] SMBIOS 2.8 present. vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] Hypervisor detected: KVM vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 vm-test-run-nixbot-gitlab> gitlab # [ 0.000001] kvm-clock: using sched offset of 537680560 cycles vm-test-run-nixbot-gitlab> gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.000005] tsc: Detected 2400.010 MHz processor vm-test-run-nixbot-gitlab> gitlab # [ 0.000817] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.000854] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs vm-test-run-nixbot-gitlab> gitlab # [ 0.000857] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT vm-test-run-nixbot-gitlab> gitlab # [ 0.000910] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.002760] found SMP MP-table at [mem 0x000f5470-0x000f547f] vm-test-run-nixbot-gitlab> gitlab # [ 0.002771] Using GB pages for direct mapping vm-test-run-nixbot-gitlab> gitlab # [ 0.002923] RAMDISK: [mem 0xbe4e0000-0xbffcffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.002930] ACPI: Early table checksum verification disabled vm-test-run-nixbot-gitlab> gitlab # [ 0.002933] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) vm-test-run-nixbot-gitlab> gitlab # [ 0.002937] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002940] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002947] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002949] ACPI: FACS 0x00000000BFFE0000 000040 vm-test-run-nixbot-gitlab> gitlab # [ 0.002951] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002953] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002954] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) vm-test-run-nixbot-gitlab> gitlab # [ 0.002956] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] vm-test-run-nixbot-gitlab> gitlab # [ 0.002957] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] vm-test-run-nixbot-gitlab> gitlab # [ 0.002957] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] vm-test-run-nixbot-gitlab> gitlab # [ 0.002958] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] vm-test-run-nixbot-gitlab> gitlab # [ 0.002958] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] vm-test-run-nixbot-gitlab> gitlab # [ 0.002959] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] vm-test-run-nixbot-gitlab> gitlab # [ 0.003440] No NUMA configuration found vm-test-run-nixbot-gitlab> gitlab # [ 0.003441] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.003444] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff] vm-test-run-nixbot-gitlab> gitlab # [ 0.006039] Zone ranges: vm-test-run-nixbot-gitlab> gitlab # [ 0.006040] DMA [mem 0x0000000000001000-0x0000000000ffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.006042] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.006044] Normal [mem 0x0000000100000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.006045] Device empty vm-test-run-nixbot-gitlab> gitlab # [ 0.006045] Movable zone start for each node vm-test-run-nixbot-gitlab> gitlab # [ 0.006046] Early memory node ranges vm-test-run-nixbot-gitlab> gitlab # [ 0.006046] node 0: [mem 0x0000000000001000-0x000000000009efff] vm-test-run-nixbot-gitlab> gitlab # [ 0.006047] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.006048] node 0: [mem 0x0000000100000000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.006049] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.006069] On node 0, zone DMA: 1 pages in unavailable ranges vm-test-run-nixbot-gitlab> gitlab # [ 0.006366] On node 0, zone DMA: 97 pages in unavailable ranges vm-test-run-nixbot-gitlab> gitlab # [ 0.128172] On node 0, zone Normal: 40 pages in unavailable ranges vm-test-run-nixbot-gitlab> gitlab # [ 0.129208] ACPI: PM-Timer IO Port: 0x608 vm-test-run-nixbot-gitlab> gitlab # [ 0.129225] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) vm-test-run-nixbot-gitlab> gitlab # [ 0.129263] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 vm-test-run-nixbot-gitlab> gitlab # [ 0.129266] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) vm-test-run-nixbot-gitlab> gitlab # [ 0.129268] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) vm-test-run-nixbot-gitlab> gitlab # [ 0.129270] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) vm-test-run-nixbot-gitlab> gitlab # [ 0.129271] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) vm-test-run-nixbot-gitlab> gitlab # [ 0.129272] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) vm-test-run-nixbot-gitlab> gitlab # [ 0.129274] ACPI: Using ACPI (MADT) for SMP configuration information vm-test-run-nixbot-gitlab> gitlab # [ 0.129275] ACPI: HPET id: 0x8086a201 base: 0xfed00000 vm-test-run-nixbot-gitlab> gitlab # [ 0.129282] TSC deadline timer available vm-test-run-nixbot-gitlab> gitlab # [ 0.129287] CPU topo: Max. logical packages: 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.129288] CPU topo: Max. logical dies: 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.129288] CPU topo: Max. dies per package: 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.129291] CPU topo: Max. threads per core: 1 vm-test-run-nixbot-gitlab> gitlab # [ 0.129292] CPU topo: Num. cores per package: 4 vm-test-run-nixbot-gitlab> gitlab # [ 0.129292] CPU topo: Num. threads per package: 4 vm-test-run-nixbot-gitlab> gitlab # [ 0.129293] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs vm-test-run-nixbot-gitlab> gitlab # [ 0.129320] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() vm-test-run-nixbot-gitlab> gitlab # [ 0.129340] kvm-guest: KVM setup pv remote TLB flush vm-test-run-nixbot-gitlab> gitlab # [ 0.129344] kvm-guest: setup PV sched yield vm-test-run-nixbot-gitlab> gitlab # [ 0.129364] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.129366] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.129367] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.129369] [mem 0xc0000000-0xfeffbfff] available for PCI devices vm-test-run-nixbot-gitlab> gitlab # [ 0.129371] Booting paravirtualized kernel on KVM vm-test-run-nixbot-gitlab> gitlab # [ 0.129375] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.133846] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 vm-test-run-nixbot-gitlab> gitlab # [ 0.136838] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 vm-test-run-nixbot-gitlab> gitlab # [ 0.136896] kvm-guest: PV spinlocks enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.136898] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.136901] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5r5nssc2dlvpnlmpdafhh1xdx2dv0q9n-nixos-system-gitlab-test/init regInfo=/nix/store/zr2rs44m0sag591c92993637bihz9h43-closure-info/registration console=ttyS0,115200n8 console=tty0 vm-test-run-nixbot-gitlab> gitlab # [ 0.136999] Unknown kernel command line parameters "regInfo=/nix/store/zr2rs44m0sag591c92993637bihz9h43-closure-info/registration", will be passed to user space. vm-test-run-nixbot-gitlab> gitlab # [ 0.137013] random: crng init done vm-test-run-nixbot-gitlab> gitlab # [ 0.137013] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes vm-test-run-nixbot-gitlab> gitlab # [ 0.147080] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.151972] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.152056] software IO TLB: area num 4. vm-test-run-nixbot-gitlab> gitlab # [ 0.237341] Fallback order for Node 0: 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.237351] Built 1 zonelists, mobility grouping on. Total pages: 1572726 vm-test-run-nixbot-gitlab> gitlab # [ 0.237353] Policy zone: Normal vm-test-run-nixbot-gitlab> gitlab # [ 0.240648] mem auto-init: stack:all(zero), heap alloc:on, heap free:off vm-test-run-nixbot-gitlab> gitlab # [ 0.251135] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 vm-test-run-nixbot-gitlab> gitlab # [ 0.266518] allocated 12582912 bytes of page_ext vm-test-run-nixbot-gitlab> gitlab # [ 0.276952] ftrace: allocating 48565 entries in 192 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.276954] ftrace: allocated 192 pages with 2 groups vm-test-run-nixbot-gitlab> gitlab # [ 0.277847] Dynamic Preempt: lazy vm-test-run-nixbot-gitlab> gitlab # [ 0.278049] rcu: Preemptible hierarchical RCU implementation. vm-test-run-nixbot-gitlab> gitlab # [ 0.278050] rcu: RCU event tracing is enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.278051] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.278052] Trampoline variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.278053] Rude variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.278053] Tracing variant of Tasks RCU enabled. vm-test-run-nixbot-gitlab> gitlab # [ 0.278054] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. vm-test-run-nixbot-gitlab> gitlab # [ 0.278054] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 vm-test-run-nixbot-gitlab> gitlab # [ 0.278082] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.278084] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.278085] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. vm-test-run-nixbot-gitlab> gitlab # [ 0.282536] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 vm-test-run-nixbot-gitlab> gitlab # [ 0.282884] rcu: srcu_init: Setting srcu_struct sizes based on contention. vm-test-run-nixbot-gitlab> gitlab # [ 0.283012] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) vm-test-run-nixbot-gitlab> gitlab # [ 0.290352] Console: colour VGA+ 80x25 vm-test-run-nixbot-gitlab> gitlab # [ 0.290355] printk: legacy console [tty0] enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.334089] printk: legacy console [ttyS0] enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.536785] ACPI: Core revision 20250807 vm-test-run-nixbot-gitlab> gitlab # [ 0.538483] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.541386] APIC: Switch to symmetric I/O mode setup vm-test-run-nixbot-gitlab> gitlab # [ 0.543145] x2apic enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.544395] APIC: Switched APIC routing to: physical x2apic vm-test-run-nixbot-gitlab> gitlab # [ 0.546098] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() vm-test-run-nixbot-gitlab> gitlab # [ 0.548307] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() vm-test-run-nixbot-gitlab> gitlab # [ 0.551065] kvm-guest: setup PV IPIs vm-test-run-nixbot-gitlab> gitlab # [ 0.553457] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 vm-test-run-nixbot-gitlab> gitlab # [ 0.555282] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.558379] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400010) vm-test-run-nixbot-gitlab> gitlab # [ 0.560487] x86/cpu: User Mode Instruction Prevention (UMIP) activated vm-test-run-nixbot-gitlab> gitlab # [ 0.561534] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 vm-test-run-nixbot-gitlab> gitlab # [ 0.563372] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.564376] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto vm-test-run-nixbot-gitlab> gitlab # [ 0.566373] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl vm-test-run-nixbot-gitlab> gitlab # [ 0.568372] Transient Scheduler Attacks: Mitigation: Clear CPU buffers vm-test-run-nixbot-gitlab> gitlab # [ 0.569372] Spectre V2 : Mitigation: Enhanced / Automatic IBRS vm-test-run-nixbot-gitlab> gitlab # [ 0.570372] Speculative Return Stack Overflow: Mitigation: Safe RET vm-test-run-nixbot-gitlab> gitlab # [ 0.571372] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization vm-test-run-nixbot-gitlab> gitlab # [ 0.572379] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier vm-test-run-nixbot-gitlab> gitlab # [ 0.574372] active return thunk: srso_alias_return_thunk vm-test-run-nixbot-gitlab> gitlab # [ 0.575401] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' vm-test-run-nixbot-gitlab> gitlab # [ 0.577372] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' vm-test-run-nixbot-gitlab> gitlab # [ 0.578372] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' vm-test-run-nixbot-gitlab> gitlab # [ 0.579372] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' vm-test-run-nixbot-gitlab> gitlab # [ 0.580372] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' vm-test-run-nixbot-gitlab> gitlab # [ 0.582372] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' vm-test-run-nixbot-gitlab> gitlab # [ 0.584371] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' vm-test-run-nixbot-gitlab> gitlab # [ 0.586372] x86/fpu: Supporting XSAVE feature 0x800: 'Control-flow User registers' vm-test-run-nixbot-gitlab> gitlab # [ 0.587372] x86/fpu: Supporting XSAVE feature 0x1000: 'Control-flow Kernel registers (KVM only)' vm-test-run-nixbot-gitlab> gitlab # [ 0.588372] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 vm-test-run-nixbot-gitlab> gitlab # [ 0.589372] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 vm-test-run-nixbot-gitlab> gitlab # [ 0.591372] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 vm-test-run-nixbot-gitlab> gitlab # [ 0.593372] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 vm-test-run-nixbot-gitlab> gitlab # [ 0.594372] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 vm-test-run-nixbot-gitlab> gitlab # [ 0.595372] x86/fpu: xstate_offset[11]: 2440, xstate_sizes[11]: 16 vm-test-run-nixbot-gitlab> gitlab # [ 0.596372] x86/fpu: xstate_offset[12]: 2456, xstate_sizes[12]: 24 vm-test-run-nixbot-gitlab> gitlab # [ 0.597372] x86/fpu: Enabled xstate features 0x1ae7, context size is 2480 bytes, using 'compacted' format. vm-test-run-nixbot-gitlab> gitlab # [ 0.627733] Freeing SMP alternatives memory: 44K vm-test-run-nixbot-gitlab> gitlab # [ 0.628375] pid_max: default: 32768 minimum: 301 vm-test-run-nixbot-gitlab> gitlab # [ 0.630488] LSM: initializing lsm=capability,landlock,yama,bpf,ima vm-test-run-nixbot-gitlab> gitlab # [ 0.631479] landlock: Up and running. vm-test-run-nixbot-gitlab> gitlab # [ 0.632372] Yama: becoming mindful. vm-test-run-nixbot-gitlab> gitlab # [ 0.633583] LSM support for eBPF active vm-test-run-nixbot-gitlab> gitlab # [ 0.635639] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.636537] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.640036] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) vm-test-run-nixbot-gitlab> gitlab # [ 0.641002] Performance Events: Fam17h+ core perfctr, AMD PMU driver. vm-test-run-nixbot-gitlab> gitlab # [ 0.641377] ... version: 2 vm-test-run-nixbot-gitlab> gitlab # [ 0.642374] ... bit width: 48 vm-test-run-nixbot-gitlab> gitlab # [ 0.643433] ... generic counters: 6 vm-test-run-nixbot-gitlab> gitlab # [ 0.644373] ... generic bitmap: 000000000000003f vm-test-run-nixbot-gitlab> gitlab # [ 0.645373] ... fixed-purpose counters: 0 vm-test-run-nixbot-gitlab> gitlab # [ 0.646373] ... fixed-purpose bitmap: 0000000000000000 vm-test-run-nixbot-gitlab> gitlab # [ 0.647374] ... value mask: 0000ffffffffffff vm-test-run-nixbot-gitlab> gitlab # [ 0.648373] ... max period: 00007fffffffffff vm-test-run-nixbot-gitlab> gitlab # [ 0.649373] ... global_ctrl mask: 000000000000003f vm-test-run-nixbot-gitlab> gitlab # [ 0.650500] signal: max sigframe size: 3376 vm-test-run-nixbot-gitlab> gitlab # [ 0.651498] rcu: Hierarchical SRCU implementation. vm-test-run-nixbot-gitlab> gitlab # [ 0.652378] rcu: Max phase no-delay instances is 400. vm-test-run-nixbot-gitlab> gitlab # [ 0.653591] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level vm-test-run-nixbot-gitlab> gitlab # [ 0.658998] smp: Bringing up secondary CPUs ... vm-test-run-nixbot-gitlab> gitlab # [ 0.659811] smpboot: x86: Booting SMP configuration: vm-test-run-nixbot-gitlab> gitlab # [ 0.660384] .... node #0, CPUs: #1 #2 #3 vm-test-run-nixbot-gitlab> gitlab # [ 0.664484] smp: Brought up 1 node, 4 CPUs vm-test-run-nixbot-gitlab> gitlab # [ 0.666374] smpboot: Total of 4 processors activated (19200.08 BogoMIPS) vm-test-run-nixbot-gitlab> gitlab # [ 0.667908] Memory: 6013324K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258300K reserved, 0K cma-reserved) vm-test-run-nixbot-gitlab> gitlab # [ 0.669853] devtmpfs: initialized vm-test-run-nixbot-gitlab> gitlab # [ 0.671398] x86/mm: Memory block size: 128MB vm-test-run-nixbot-gitlab> gitlab # [ 0.674591] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns vm-test-run-nixbot-gitlab> gitlab # [ 0.675428] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 0.676464] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). vm-test-run-nixbot-gitlab> gitlab # [ 0.677509] pinctrl core: initialized pinctrl subsystem vm-test-run-nixbot-gitlab> gitlab # [ 0.678786] PM: RTC time: 09:13:24, date: 2026-06-16 vm-test-run-nixbot-gitlab> gitlab # [ 0.682601] NET: Registered PF_NETLINK/PF_ROUTE protocol family vm-test-run-nixbot-gitlab> gitlab # [ 0.684887] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.685451] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.687597] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations vm-test-run-nixbot-gitlab> gitlab # [ 0.688389] audit: initializing netlink subsys (disabled) vm-test-run-nixbot-gitlab> gitlab # [ 0.689474] audit: type=2000 audit(1781601204.847:1): state=initialized audit_enabled=0 res=1 vm-test-run-nixbot-gitlab> gitlab # [ 0.689712] thermal_sys: Registered thermal governor 'fair_share' vm-test-run-nixbot-gitlab> gitlab # [ 0.691374] thermal_sys: Registered thermal governor 'bang_bang' vm-test-run-nixbot-gitlab> gitlab # [ 0.692382] thermal_sys: Registered thermal governor 'step_wise' vm-test-run-nixbot-gitlab> gitlab # [ 0.693374] thermal_sys: Registered thermal governor 'user_space' vm-test-run-nixbot-gitlab> gitlab # [ 0.694374] thermal_sys: Registered thermal governor 'power_allocator' vm-test-run-nixbot-gitlab> gitlab # [ 0.695392] cpuidle: using governor menu vm-test-run-nixbot-gitlab> gitlab # [ 0.698666] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 vm-test-run-nixbot-gitlab> gitlab # [ 0.699741] PCI: Using configuration type 1 for base access vm-test-run-nixbot-gitlab> gitlab # [ 0.700374] PCI: Using configuration type 1 for extended access vm-test-run-nixbot-gitlab> gitlab # [ 0.702119] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. vm-test-run-nixbot-gitlab> gitlab # [ 0.704823] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.706374] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.707375] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages vm-test-run-nixbot-gitlab> gitlab # [ 0.708374] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page vm-test-run-nixbot-gitlab> gitlab # [ 0.713590] ACPI: Added _OSI(Module Device) vm-test-run-nixbot-gitlab> gitlab # [ 0.717384] ACPI: Added _OSI(Processor Device) vm-test-run-nixbot-gitlab> gitlab # [ 0.720383] ACPI: Added _OSI(Processor Aggregator Device) vm-test-run-nixbot-gitlab> gitlab # [ 0.729403] ACPI: 1 ACPI AML tables successfully acquired and loaded vm-test-run-nixbot-gitlab> gitlab # [ 0.737472] ACPI: Interpreter enabled vm-test-run-nixbot-gitlab> gitlab # [ 0.738499] ACPI: PM: (supports S0 S3 S4 S5) vm-test-run-nixbot-gitlab> gitlab # [ 0.739377] ACPI: Using IOAPIC for interrupt routing vm-test-run-nixbot-gitlab> gitlab # [ 0.741526] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug vm-test-run-nixbot-gitlab> gitlab # [ 0.744374] PCI: Using E820 reservations for host bridge windows vm-test-run-nixbot-gitlab> gitlab # [ 0.745608] ACPI: Enabled 2 GPEs in block 00 to 0F vm-test-run-nixbot-gitlab> gitlab # [ 0.752806] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) vm-test-run-nixbot-gitlab> gitlab # [ 0.754381] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] vm-test-run-nixbot-gitlab> gitlab # [ 0.758005] acpiphp: Slot [3] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.759457] acpiphp: Slot [4] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.760453] acpiphp: Slot [5] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.761453] acpiphp: Slot [6] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.763435] acpiphp: Slot [7] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.764468] acpiphp: Slot [8] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.765443] acpiphp: Slot [9] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.767444] acpiphp: Slot [10] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.768444] acpiphp: Slot [11] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.769459] acpiphp: Slot [12] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.771442] acpiphp: Slot [13] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.772409] acpiphp: Slot [14] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.773409] acpiphp: Slot [15] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.774429] acpiphp: Slot [16] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.776409] acpiphp: Slot [17] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.777409] acpiphp: Slot [18] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.778431] acpiphp: Slot [19] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.780410] acpiphp: Slot [20] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.781407] acpiphp: Slot [21] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.782409] acpiphp: Slot [22] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.784425] acpiphp: Slot [23] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.785410] acpiphp: Slot [24] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.786409] acpiphp: Slot [25] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.787408] acpiphp: Slot [26] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.789432] acpiphp: Slot [27] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.790408] acpiphp: Slot [28] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.791409] acpiphp: Slot [29] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.793408] acpiphp: Slot [30] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.794434] acpiphp: Slot [31] registered vm-test-run-nixbot-gitlab> gitlab # [ 0.795581] PCI host bridge to bus 0000:00 vm-test-run-nixbot-gitlab> gitlab # [ 0.799589] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] vm-test-run-nixbot-gitlab> gitlab # [ 0.804386] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] vm-test-run-nixbot-gitlab> gitlab # [ 0.809386] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] vm-test-run-nixbot-gitlab> gitlab # [ 0.814385] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] vm-test-run-nixbot-gitlab> gitlab # [ 0.819385] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] vm-test-run-nixbot-gitlab> gitlab # [ 0.825424] pci_bus 0000:00: root bus resource [bus 00-ff] vm-test-run-nixbot-gitlab> gitlab # [ 0.830783] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.839943] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.847233] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.856542] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] vm-test-run-nixbot-gitlab> gitlab # [ 0.859515] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk vm-test-run-nixbot-gitlab> gitlab # [ 0.863375] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk vm-test-run-nixbot-gitlab> gitlab # [ 0.865375] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk vm-test-run-nixbot-gitlab> gitlab # [ 0.867386] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk vm-test-run-nixbot-gitlab> gitlab # [ 0.872474] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.882546] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] vm-test-run-nixbot-gitlab> gitlab # [ 0.890762] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.898019] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI vm-test-run-nixbot-gitlab> gitlab # [ 0.903420] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB vm-test-run-nixbot-gitlab> gitlab # [ 0.909580] pci 0000:00:01.3: quirk_piix4_acpi+0x0/0x180 took 10742 usecs vm-test-run-nixbot-gitlab> gitlab # [ 0.915259] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.933475] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.935427] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.938532] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 0.945016] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] vm-test-run-nixbot-gitlab> gitlab # [ 0.980392] pci 0000:00:02.0: pci_fixup_video+0x0/0x100 took 35156 usecs vm-test-run-nixbot-gitlab> gitlab # [ 0.986977] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 0.994412] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] vm-test-run-nixbot-gitlab> gitlab # [ 0.995395] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] vm-test-run-nixbot-gitlab> gitlab # [ 0.997486] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.002422] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.013274] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.027418] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] vm-test-run-nixbot-gitlab> gitlab # [ 1.031418] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.035516] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.045524] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.056410] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] vm-test-run-nixbot-gitlab> gitlab # [ 1.059415] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.060431] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.065776] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.071409] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] vm-test-run-nixbot-gitlab> gitlab # [ 1.073390] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.075429] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.079453] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.085406] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] vm-test-run-nixbot-gitlab> gitlab # [ 1.086389] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.088428] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.092452] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.098403] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] vm-test-run-nixbot-gitlab> gitlab # [ 1.099389] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.101430] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.106709] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.110389] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] vm-test-run-nixbot-gitlab> gitlab # [ 1.112390] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.114427] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.116390] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.123795] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.135450] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.139539] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.150508] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.163432] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] vm-test-run-nixbot-gitlab> gitlab # [ 1.166410] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] vm-test-run-nixbot-gitlab> gitlab # [ 1.170497] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.178220] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint vm-test-run-nixbot-gitlab> gitlab # [ 1.184390] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] vm-test-run-nixbot-gitlab> gitlab # [ 1.186389] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] vm-test-run-nixbot-gitlab> gitlab # [ 1.187430] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] vm-test-run-nixbot-gitlab> gitlab # [ 1.210968] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 vm-test-run-nixbot-gitlab> gitlab # [ 1.215810] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 vm-test-run-nixbot-gitlab> gitlab # [ 1.219782] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 vm-test-run-nixbot-gitlab> gitlab # [ 1.222774] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 vm-test-run-nixbot-gitlab> gitlab # [ 1.226488] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 vm-test-run-nixbot-gitlab> gitlab # [ 1.230646] iommu: Default domain type: Translated vm-test-run-nixbot-gitlab> gitlab # [ 1.232374] iommu: DMA domain TLB invalidation policy: lazy mode vm-test-run-nixbot-gitlab> gitlab # [ 1.234736] ACPI: bus type USB registered vm-test-run-nixbot-gitlab> gitlab # [ 1.235436] usbcore: registered new interface driver usbfs vm-test-run-nixbot-gitlab> gitlab # [ 1.237407] usbcore: registered new interface driver hub vm-test-run-nixbot-gitlab> gitlab # [ 1.239386] usbcore: registered new device driver usb vm-test-run-nixbot-gitlab> gitlab # [ 1.241879] NetLabel: Initializing vm-test-run-nixbot-gitlab> gitlab # [ 1.243375] NetLabel: domain hash size = 128 vm-test-run-nixbot-gitlab> gitlab # [ 1.244374] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO vm-test-run-nixbot-gitlab> gitlab # [ 1.246442] NetLabel: unlabeled traffic allowed by default vm-test-run-nixbot-gitlab> gitlab # [ 1.247459] PCI: Using ACPI for IRQ routing vm-test-run-nixbot-gitlab> gitlab # [ 1.250612] pci 0000:00:02.0: vgaarb: setting as boot VGA device vm-test-run-nixbot-gitlab> gitlab # [ 1.251369] pci 0000:00:02.0: vgaarb: bridge control possible vm-test-run-nixbot-gitlab> gitlab # [ 1.251369] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none vm-test-run-nixbot-gitlab> gitlab # [ 1.263394] vgaarb: loaded vm-test-run-nixbot-gitlab> gitlab # [ 1.266026] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 vm-test-run-nixbot-gitlab> gitlab # [ 1.269399] hpet0: 3 comparators, 64-bit 100.000000 MHz counter vm-test-run-nixbot-gitlab> gitlab # [ 1.278789] clocksource: Switched to clocksource kvm-clock vm-test-run-nixbot-gitlab> gitlab # [ 1.285185] VFS: Disk quotas dquot_6.6.0 vm-test-run-nixbot-gitlab> gitlab # [ 1.287779] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) vm-test-run-nixbot-gitlab> gitlab # [ 1.292512] pnp: PnP ACPI init vm-test-run-nixbot-gitlab> gitlab # [ 1.295899] pnp: PnP ACPI: found 6 devices vm-test-run-nixbot-gitlab> gitlab # [ 1.306300] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns vm-test-run-nixbot-gitlab> gitlab # [ 1.311595] clocksource: Switched to clocksource acpi_pm vm-test-run-nixbot-gitlab> gitlab # [ 1.315136] NET: Registered PF_INET protocol family vm-test-run-nixbot-gitlab> gitlab # [ 1.320461] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.340962] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.343562] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.345968] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.350950] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.353289] TCP: Hash tables configured (established 65536 bind 65536) vm-test-run-nixbot-gitlab> gitlab # [ 1.355395] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.357720] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.360059] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) vm-test-run-nixbot-gitlab> gitlab # [ 1.362354] NET: Registered PF_UNIX/PF_LOCAL protocol family vm-test-run-nixbot-gitlab> gitlab # [ 1.364151] NET: Registered PF_XDP protocol family vm-test-run-nixbot-gitlab> gitlab # [ 1.365692] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] vm-test-run-nixbot-gitlab> gitlab # [ 1.367547] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] vm-test-run-nixbot-gitlab> gitlab # [ 1.369593] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] vm-test-run-nixbot-gitlab> gitlab # [ 1.374726] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] vm-test-run-nixbot-gitlab> gitlab # [ 1.379861] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] vm-test-run-nixbot-gitlab> gitlab # [ 1.385951] pci 0000:00:01.0: PIIX3: Enabling Passive Release vm-test-run-nixbot-gitlab> gitlab # [ 1.390577] pci 0000:00:00.0: Limiting direct PCI/PCI transfers vm-test-run-nixbot-gitlab> gitlab # [ 1.399607] ACPI: \_SB_.LNKD: Enabled at IRQ 11 vm-test-run-nixbot-gitlab> gitlab # [ 1.408256] pci 0000:00:01.2: quirk_usb_early_handoff+0x0/0x7c0 took 12842 usecs vm-test-run-nixbot-gitlab> gitlab # [ 1.414399] PCI: CLS 0 bytes, default 64 vm-test-run-nixbot-gitlab> gitlab # [ 1.417912] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) vm-test-run-nixbot-gitlab> gitlab # [ 1.418458] Trying to unpack rootfs image as initramfs... vm-test-run-nixbot-gitlab> gitlab # [ 1.418843] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB) vm-test-run-nixbot-gitlab> gitlab # [ 1.431119] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229840d1729, max_idle_ns: 440795276167 ns vm-test-run-nixbot-gitlab> gitlab # [ 1.453676] Initialise system trusted keyrings vm-test-run-nixbot-gitlab> gitlab # [ 1.455724] workingset: timestamp_bits=40 max_order=21 bucket_order=0 vm-test-run-nixbot-gitlab> gitlab # [ 1.468593] Key type asymmetric registered vm-test-run-nixbot-gitlab> gitlab # [ 1.469900] Asymmetric key parser 'x509' registered vm-test-run-nixbot-gitlab> gitlab # [ 1.471523] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) vm-test-run-nixbot-gitlab> gitlab # [ 1.474242] io scheduler mq-deadline registered vm-test-run-nixbot-gitlab> gitlab # [ 1.475706] io scheduler kyber registered vm-test-run-nixbot-gitlab> gitlab # [ 1.479951] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled vm-test-run-nixbot-gitlab> gitlab # [ 1.482236] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A vm-test-run-nixbot-gitlab> gitlab # [ 1.486284] Linux agpgart interface v0.103 vm-test-run-nixbot-gitlab> gitlab # [ 1.487711] ACPI: bus type drm_connector registered vm-test-run-nixbot-gitlab> gitlab # [ 1.492016] usbcore: registered new interface driver usbserial_generic vm-test-run-nixbot-gitlab> gitlab # [ 1.493952] usbserial: USB Serial support registered for generic vm-test-run-nixbot-gitlab> gitlab # [ 1.495782] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. vm-test-run-nixbot-gitlab> gitlab # [ 1.495782] Please enable it if your BIOS has the CPPC option. vm-test-run-nixbot-gitlab> gitlab # [ 1.499767] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled vm-test-run-nixbot-gitlab> gitlab # [ 1.502165] drop_monitor: Initializing network drop monitor service vm-test-run-nixbot-gitlab> gitlab # [ 1.504290] NET: Registered PF_INET6 protocol family vm-test-run-nixbot-gitlab> gitlab # [ 1.507280] Segment Routing with IPv6 vm-test-run-nixbot-gitlab> gitlab # [ 1.508551] In-situ OAM (IOAM) with IPv6 vm-test-run-nixbot-gitlab> gitlab # [ 1.514392] IPI shorthand broadcast: enabled vm-test-run-nixbot-gitlab> gitlab # [ 1.520122] sched_clock: Marking stable (1256030585, 263475068)->(1601887118, -82381465) vm-test-run-nixbot-gitlab> gitlab # [ 1.523114] registered taskstats version 1 vm-test-run-nixbot-gitlab> gitlab # [ 1.524931] Loading compiled-in X.509 certificates vm-test-run-nixbot-gitlab> gitlab # [ 1.536079] Demotion targets for Node 0: null vm-test-run-nixbot-gitlab> gitlab # [ 1.538007] Key type .fscrypt registered vm-test-run-nixbot-gitlab> gitlab # [ 1.539288] Key type fscrypt-provisioning registered vm-test-run-nixbot-gitlab> gitlab # [ 1.541044] ima: No TPM chip found, activating TPM-bypass! vm-test-run-nixbot-gitlab> gitlab # [ 1.543267] ima: Allocated hash algorithm: sha1 vm-test-run-nixbot-gitlab> gitlab # [ 1.544793] ima: No architecture policies found vm-test-run-nixbot-gitlab> gitlab # [ 1.546756] PM: Magic number: 14:323:222 vm-test-run-nixbot-gitlab> gitlab # [ 1.549132] RAS: Correctable Errors collector initialized. vm-test-run-nixbot-gitlab> gitlab # [ 1.554559] clk: Disabling unused clocks vm-test-run-nixbot-gitlab> gitlab # [ 1.555839] PM: genpd: Disabling unused power domains vm-test-run-nixbot-gitlab> gitlab # [ 1.741537] Freeing initrd memory: 27584K vm-test-run-nixbot-gitlab> gitlab # [ 1.746068] Freeing unused decrypted memory: 2028K vm-test-run-nixbot-gitlab> gitlab # [ 1.749980] Freeing unused kernel image (initmem) memory: 3644K vm-test-run-nixbot-gitlab> gitlab # [ 1.751944] Write protecting the kernel read-only data: 32768k vm-test-run-nixbot-gitlab> gitlab # [ 1.755254] Freeing unused kernel image (text/rodata gap) memory: 1332K vm-test-run-nixbot-gitlab> gitlab # [ 1.758042] Freeing unused kernel image (rodata/data gap) memory: 816K vm-test-run-nixbot-gitlab> gitlab # [ 1.810424] x86/mm: Checked W+X mappings: passed, no W+X pages found. vm-test-run-nixbot-gitlab> gitlab # [ 1.812517] Run /init as init process vm-test-run-nixbot-gitlab> gitlab # [ 1.830321] systemd[1]: Inserted module 'autofs4' vm-test-run-nixbot-gitlab> gitlab # [ 1.861003] fuse: init (API version 7.45) vm-test-run-nixbot-gitlab> gitlab # [ 1.883903] ACPI: \_SB_.LNKC: Enabled at IRQ 10 vm-test-run-nixbot-gitlab> gitlab # [ 1.894995] ACPI: \_SB_.LNKA: Enabled at IRQ 10 vm-test-run-nixbot-gitlab> gitlab # [ 1.900235] ACPI: \_SB_.LNKB: Enabled at IRQ 11 vm-test-run-nixbot-gitlab> gitlab # [ 1.952144] systemd[1]: Successfully made /usr/ read-only. vm-test-run-nixbot-gitlab> gitlab # [ 2.296417] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) vm-test-run-nixbot-gitlab> gitlab # [ 2.318452] systemd[1]: Detected virtualization kvm. vm-test-run-nixbot-gitlab> gitlab # [ 2.322611] systemd[1]: Detected architecture x86-64. vm-test-run-nixbot-gitlab> gitlab # [ 2.326616] systemd[1]: Running in initrd. vm-test-run-nixbot-gitlab> gitlab # [ 2.330940] systemd[1]: Initializing machine ID from random generator. vm-test-run-nixbot-gitlab> gitlab # [ 2.336330] systemd[1]: Hostname set to . vm-test-run-nixbot-gitlab> gitlab # [ 2.403855] systemd[1]: Queued start job for default target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 2.567652] systemd[1]: Created slice Slice /system/modprobe. vm-test-run-nixbot-gitlab> gitlab # [ 2.569864] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 2.574693] systemd[1]: Expecting device /dev/disk/by-label/nix-store... vm-test-run-nixbot-gitlab> gitlab # [ 2.579928] systemd[1]: Expecting device /dev/disk/by-label/nixos... vm-test-run-nixbot-gitlab> gitlab # [ 2.585028] systemd[1]: Reached target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 2.589110] systemd[1]: Reached target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 2.592773] systemd[1]: Reached target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 2.595711] systemd[1]: Reached target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 2.599182] systemd[1]: Listening on D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 2.603668] systemd[1]: Listening on Journal Socket (/dev/log). vm-test-run-nixbot-gitlab> gitlab # [ 2.607263] systemd[1]: Listening on Journal Sockets. vm-test-run-nixbot-gitlab> gitlab # [ 2.609086] systemd[1]: Listening on udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 2.610933] systemd[1]: Listening on udev Kernel Socket. vm-test-run-nixbot-gitlab> gitlab # [ 2.612696] systemd[1]: Reached target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 2.618899] systemd[1]: Starting Create List of Static Device Nodes... vm-test-run-nixbot-gitlab> gitlab # [ 2.624809] systemd[1]: Starting Load Kernel Module 9pnet_virtio... vm-test-run-nixbot-gitlab> gitlab # [ 2.635073] systemd[1]: Starting Load Kernel Module configfs... vm-test-run-nixbot-gitlab> gitlab # [ 2.640316] systemd[1]: Starting Journal Service... vm-test-run-nixbot-gitlab> gitlab # [ 2.652651] systemd[1]: Starting Load Kernel Modules... vm-test-run-nixbot-gitlab> gitlab # [ 2.654409] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 2.662625] systemd[1]: Starting Coldplug All udev Devices... vm-test-run-nixbot-gitlab> gitlab # [ 2.671613] systemd[1]: Finished Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 2.674841] systemd[1]: modprobe@configfs.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 2.678923] systemd[1]: Finished Load Kernel Module configfs. vm-test-run-nixbot-gitlab> gitlab # [ 2.681327] netfs: FS-Cache loaded vm-test-run-nixbot-gitlab> gitlab # [ 2.683874] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config vm-test-run-nixbot-gitlab> gitlab # [ 2.691544] systemd-journald[150]: Collecting audit messages is disabled. vm-test-run-nixbot-gitlab> gitlab # [ 2.696234] 9pnet: Installing 9P2000 support vm-test-run-nixbot-gitlab> gitlab # [ 2.701152] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... vm-test-run-nixbot-gitlab> gitlab # [ 2.707523] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 2.711236] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. vm-test-run-nixbot-gitlab> gitlab # [ 2.715580] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev vm-test-run-nixbot-gitlab> gitlab # [ 2.728192] systemd[1]: Finished Load Kernel Module 9pnet_virtio. vm-test-run-nixbot-gitlab> gitlab # [ 2.734864] systemd[1]: Finished Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 2.741635] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 2.746918] systemd[1]: Starting Apply Kernel Variables... vm-test-run-nixbot-gitlab> gitlab # [ 2.759847] systemd[1]: Starting Create Static Device Nodes in /dev... vm-test-run-nixbot-gitlab> gitlab # [ 2.500331] systemd-modules-load[154]: Using 4 probe threads vm-test-run-nixbot-gitlab> gitlab # [ 2.502628] systemd-modules-load[154]: Insert[ 2.766166] systemd[1]: Started Journal Service. vm-test-run-nixbot-gitlab> gitlab # ed module 'virtio_balloon' vm-test-run-nixbot-gitlab> gitlab # [ 2.505553] systemd-modules-load[154]: Inserted module 'virtio_gpu' vm-test-run-nixbot-gitlab> gitlab # [ 2.507398] systemd-modules-load[154]: Inserted module 'dm_mod' vm-test-run-nixbot-gitlab> gitlab # [ 2.510663] systemd[1]: Finished Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 2.516652] systemd[1]: Finished Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 2.519982] systemd[1]: Reached target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 2.521800] systemd[1]: Reached target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 2.523302] systemd[1]: Starting Create System Files and Directories... vm-test-run-nixbot-gitlab> gitlab # [ 2.525498] systemd[1]: Starting Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 2.541404] systemd[1]: Finished Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 2.550649] systemd-udevd[192]: Using default interface naming scheme 'v260'. vm-test-run-nixbot-gitlab> gitlab # [ 2.567286] systemd[1]: Started Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 2.594193] systemd[1]: Finished Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 2.598205] systemd[1]: Reached target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 2.601554] systemd[1]: Reached target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 2.999631] SCSI subsystem initialized vm-test-run-nixbot-gitlab> gitlab # [ 3.008969] uhci_hcd 0000:00:01.2: UHCI Host Controller vm-test-run-nixbot-gitlab> gitlab # [ 3.011641] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 vm-test-run-nixbot-gitlab> gitlab # [ 3.014876] uhci_hcd 0000:00:01.2: detected 2 ports vm-test-run-nixbot-gitlab> gitlab # [ 3.015434] virtio_blk virtio4: 4/0/0 default/read/poll queues vm-test-run-nixbot-gitlab> gitlab # [ 3.017443] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 vm-test-run-nixbot-gitlab> gitlab # [ 3.021044] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 vm-test-run-nixbot-gitlab> gitlab # [ 3.023410] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 vm-test-run-nixbot-gitlab> gitlab # [ 3.025899] usb usb1: Product: UHCI Host Controller vm-test-run-nixbot-gitlab> gitlab # [ 3.027541] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd vm-test-run-nixbot-gitlab> gitlab # [ 3.029544] usb usb1: SerialNumber: 0000:00:01.2 vm-test-run-nixbot-gitlab> gitlab # [ 3.032639] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) vm-test-run-nixbot-gitlab> gitlab # [ 3.038657] hub 1-0:1.0: USB hub found vm-test-run-nixbot-gitlab> gitlab # [ 3.040036] hub 1-0:1.0: 2 ports detected vm-test-run-nixbot-gitlab> gitlab # [ 3.040120] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 vm-test-run-nixbot-gitlab> gitlab # [ 3.047361] serio: i8042 KBD port at 0x60,0x64 irq 1 vm-test-run-nixbot-gitlab> gitlab # [ 3.048895] serio: i8042 AUX port at 0x60,0x64 irq 12 vm-test-run-nixbot-gitlab> gitlab # [ 3.054101] virtio_blk virtio5: 4/0/0 default/read/poll queues vm-test-run-nixbot-gitlab> gitlab # [ 3.056245] scsi host0: ata_piix vm-test-run-nixbot-gitlab> gitlab # [ 3.058702] scsi host1: ata_piix vm-test-run-nixbot-gitlab> gitlab # [ 3.059748] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 vm-test-run-nixbot-gitlab> gitlab # [ 3.061760] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 vm-test-run-nixbot-gitlab> gitlab # [ 3.063052] virtio_blk virtio5: [vdb] 11486408 512-byte logical blocks (5.88 GB/5.48 GiB) vm-test-run-nixbot-gitlab> gitlab # [ 2.842428] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 2.876192] systemd-vconsole-setup[232]: Configuration of first virtual console was skipped, ignoring remaining ones. vm-test-run-nixbot-gitlab> gitlab # [ 2.879096] systemd[1]: Finished Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 3.218725] ata2: found unknown device (class 0) vm-test-run-nixbot-gitlab> gitlab # [ 3.222975] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 vm-test-run-nixbot-gitlab> gitlab # [ 3.230470] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 vm-test-run-nixbot-gitlab> gitlab # [ 3.265661] usb 1-1: new full-speed USB device number 2 using uhci_hcd vm-test-run-nixbot-gitlab> gitlab # [ 3.274743] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 vm-test-run-nixbot-gitlab> gitlab # [ 3.016688] (udev-worker)[204]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. vm-test-run-nixbot-gitlab> gitlab # [ 3.028961] (udev-worker)[204]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 3.035996] (udev-worker)[200]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 3.046189] systemd[1]: Found device /dev/disk/by-label/nixos. vm-test-run-nixbot-gitlab> gitlab # [ 3.051148] systemd[1]: Found device /dev/disk/by-label/nix-store. vm-test-run-nixbot-gitlab> gitlab # [ 3.054605] systemd[1]: Reached target Initrd Root Device. vm-test-run-nixbot-gitlab> gitlab # [ 3.319840] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray vm-test-run-nixbot-gitlab> gitlab # [ 3.058791] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... vm-test-run-nixbot-gitlab> gitlab # [ 3.087605] systemd-fsck[242]: nixos: clean, 12/65536 files, 13019/262144 blocks vm-test-run-nixbot-gitlab> gitlab # [ 3.095277] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. vm-test-run-nixbot-gitlab> gitlab # [ 3.361675] cdrom: Uniform CD-ROM driver Revision: 3.20 vm-test-run-nixbot-gitlab> gitlab # [ 3.436210] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 vm-test-run-nixbot-gitlab> gitlab # [ 3.441283] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 vm-test-run-nixbot-gitlab> gitlab # [ 3.445722] usb 1-1: Product: QEMU USB Tablet vm-test-run-nixbot-gitlab> gitlab # [ 3.448605] usb 1-1: Manufacturer: QEMU vm-test-run-nixbot-gitlab> gitlab # [ 3.451149] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 vm-test-run-nixbot-gitlab> gitlab # [ 3.491669] hid: raw HID events driver (C) Jiri Kosina vm-test-run-nixbot-gitlab> gitlab # [ 3.504995] usbcore: registered new interface driver usbhid vm-test-run-nixbot-gitlab> gitlab # [ 3.507664] usbhid: USB HID core driver vm-test-run-nixbot-gitlab> gitlab # [ 3.518873] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 vm-test-run-nixbot-gitlab> gitlab # [ 3.522351] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 vm-test-run-nixbot-gitlab> gitlab # [ 3.412326] systemd[1]: Mounting /sysroot... vm-test-run-nixbot-gitlab> gitlab # [ 3.879110] EXT4-fs (vda): mounted filesystem 0fcc4280-dcbd-4bd9-bcc9-80c7af117e8e r/w with ordered data mode. Quota mode: none. vm-test-run-nixbot-gitlab> gitlab # [ 3.621339] systemd[1]: Mounted /sysroot. vm-test-run-nixbot-gitlab> gitlab # [ 3.623374] systemd[1]: Reached target Initrd Root File System. vm-test-run-nixbot-gitlab> gitlab # [ 3.630918] systemd[1]: Mounting /sysroot/nix/.ro-store... vm-test-run-nixbot-gitlab> gitlab # [ 3.646910] systemd[1]: Mounting /sysroot/nix/.rw-store... vm-test-run-nixbot-gitlab> gitlab # [ 3.652468] systemd[1]: Mounting /sysroot/run... vm-test-run-nixbot-gitlab> gitlab # [ 3.660577] systemd[1]: Mount[ 3.924525] erofs (device vdb): mounted with root inode @ nid 128. vm-test-run-nixbot-gitlab> gitlab # ing /sysroot/tmp/shared... vm-test-run-nixbot-gitlab> gitlab # [ 3.668918] systemd[1]: Mounting /sysroot/tmp/xchg... vm-test-run-nixbot-gitlab> gitlab # [ 3.675735] systemd[1]: Starting Mountpoints Configured in the Real Root... vm-test-run-nixbot-gitlab> gitlab # [ 3.679232] systemd[1]: Mounted /sysroot/nix/.ro-store. vm-test-run-nixbot-gitlab> gitlab # [ 3.945188] 9p: Installing v9fs 9p2000 file system support vm-test-run-nixbot-gitlab> gitlab # [ 3.688851] systemd[1]: Mounted /sysroot/nix/.rw-store. vm-test-run-nixbot-gitlab> gitlab # [ 3.691567] systemd[1]: Mounted /sysroot/run. vm-test-run-nixbot-gitlab> gitlab # [ 3.693308] systemd[1]: Mounted /sysroot/tmp/shared. vm-test-run-nixbot-gitlab> gitlab # [ 3.695762] systemd[1]: Starting rw-sysroot-nix-store.service... vm-test-run-nixbot-gitlab> gitlab # [ 3.697663] systemd-sysroot-fstab-check[271]: /sysroot should be mounted in the initrd, will request daemon-reload. vm-test-run-nixbot-gitlab> gitlab # [ 3.700409] systemd[1]: Mounted /sysroot/tmp/xchg. vm-test-run-nixbot-gitlab> gitlab # [ 3.701856] systemd[1]: Reload requested from client PID 271 ('systemd-sysroot') (unit initrd-parse-etc.service)... vm-test-run-nixbot-gitlab> gitlab # [ 3.704599] systemd[1]: Reloading... vm-test-run-nixbot-gitlab> gitlab # [ 4.028874] systemd[1]: Reloading finished in 327 ms. vm-test-run-nixbot-gitlab> gitlab # [ 4.095831] systemd-sysroot-fstab-check[271]: Requesting initrd-fs.target/start/replace... vm-test-run-nixbot-gitlab> gitlab # [ 4.262818] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.271703] systemd[1]: Finished rw-sysroot-nix-store.service. vm-test-run-nixbot-gitlab> gitlab # [ 4.284291] systemd-sysroot-fstab-check[271]: Requesting swap.target/start/replace... vm-test-run-nixbot-gitlab> gitlab # [ 4.294158] systemd[1]: Starting rw-sysroot-nix-store.service... vm-test-run-nixbot-gitlab> gitlab # [ 4.298794] systemd[1]: initrd-parse-etc.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.303485] systemd[1]: Finished Mountpoints Configured in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.307180] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. vm-test-run-nixbot-gitlab> gitlab # [ 4.322535] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.330818] systemd[1]: Finished rw-sysroot-nix-store.service. vm-test-run-nixbot-gitlab> gitlab # [ 4.413228] systemd[1]: Mounting /sysroot/nix/store... vm-test-run-nixbot-gitlab> gitlab # [ 4.472425] systemd[1]: Mounted /sysroot/nix/store. vm-test-run-nixbot-gitlab> gitlab # [ 4.477673] systemd[1]: Reached target Initrd File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 4.487420] systemd[1]: Starting Find NixOS closure... vm-test-run-nixbot-gitlab> gitlab # [ 4.496858] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... vm-test-run-nixbot-gitlab> gitlab # [ 4.524092] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.528897] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.547805] systemd[1]: Finished Find NixOS closure. vm-test-run-nixbot-gitlab> gitlab # [ 4.550955] systemd[1]: Reached target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 4.556907] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... vm-test-run-nixbot-gitlab> gitlab # [ 4.595289] systemd[1]: Stopped target Initrd Default Target. vm-test-run-nixbot-gitlab> gitlab # [ 4.598841] systemd[1]: Stopped target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 4.602212] systemd[1]: Stopped target Initrd Root Device. vm-test-run-nixbot-gitlab> gitlab # [ 4.605956] systemd[1]: Stopped target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.609464] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.614234] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 4.619225] systemd[1]: Stopped target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.622859] systemd[1]: Stopped target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.625885] systemd[1]: Stopped target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 4.629322] systemd[1]: Stopped target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 4.632102] systemd[1]: Stopped target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 4.635134] systemd[1]: dbus.socket: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.638105] systemd[1]: Closed D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 4.640712] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.643952] systemd[1]: Stopped Find NixOS closure. vm-test-run-nixbot-gitlab> gitlab # [ 4.645934] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio vm-test-run-nixbot-gitlab> gitlab # [ 4.650724] systemd[1]: Starting rw-sysroot-nix-store.service... vm-test-run-nixbot-gitlab> gitlab # [ 4.653213] systemd[1]: systemd-sysctl.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.655468] systemd[1]: Stopped Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 4.656877] systemd[1]: systemd-modules-load.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.658912] systemd[1]: Stopped Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 4.660235] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.662239] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.665904] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.667713] systemd[1]: Stopped Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 4.673810] systemd[1]: Stopped target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 4.675317] systemd[1]: Stopped target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 4.677106] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.678994] systemd[1]: Stopped Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 4.683520] systemd[1]: Stopping Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 4.685652] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.687637] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 4.696897] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.700651] systemd[1]: Finished rw-sysroot-nix-store.service. vm-test-run-nixbot-gitlab> gitlab # [ 4.706504] systemd[1]: systemd-udevd.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.711369] systemd[1]: Stopped Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 4.713203] systemd[1]: systemd-udevd.service: Consumed 1.099s CPU time over 2.181s wall clock time, 24.9M memory peak. vm-test-run-nixbot-gitlab> gitlab # [ 4.719258] systemd[1]: initrd-cleanup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.722982] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. vm-test-run-nixbot-gitlab> gitlab # [ 4.731386] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.733236] systemd[1]: Closed udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 4.739772] systemd[1]: Starting Cleanup udev Database... vm-test-run-nixbot-gitlab> gitlab # [ 4.741180] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.743055] systemd[1]: Stopped Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 4.747641] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.749555] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 4.755298] systemd[1]: kmod-static-nodes.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.757095] systemd[1]: Stopped Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 4.764554] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 4.768697] systemd[1]: Finished Cleanup udev Database. vm-test-run-nixbot-gitlab> gitlab # [ 4.773862] systemd[1]: Reached target Switch Root. vm-test-run-nixbot-gitlab> gitlab # [ 4.777005] systemd[1]: Starting NixOS Activation... vm-test-run-nixbot-gitlab> gitlab # [ 4.910408] initrd-nixos-activation-start[568]: booting system configuration /nix/store/5r5nssc2dlvpnlmpdafhh1xdx2dv0q9n-nixos-system-gitlab-test vm-test-run-nixbot-gitlab> gitlab # [ 4.955913] initrd-nixos-activation-start[568]: running activation script... vm-test-run-nixbot-gitlab> gitlab # [ 5.185655] initrd-nixos-activation-start[591]: setting up /etc... vm-test-run-nixbot-gitlab> gitlab # [ 5.316646] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 5.321918] systemd[1]: Finished NixOS Activation. vm-test-run-nixbot-gitlab> gitlab # [ 5.337658] systemd[1]: Starting Switch Root... vm-test-run-nixbot-gitlab> gitlab # [ 5.358799] systemd[1]: Switching root. vm-test-run-nixbot-gitlab> gitlab # [ 6.018756] systemd-journald[150]: Received SIGTERM from PID 1 (systemd). vm-test-run-nixbot-gitlab> gitlab # [ 6.150589] NET: Registered PF_VSOCK protocol family vm-test-run-nixbot-gitlab> gitlab # [ 6.540238] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) vm-test-run-nixbot-gitlab> gitlab # [ 6.556598] systemd[1]: Detected virtualization kvm. vm-test-run-nixbot-gitlab> gitlab # [ 6.559676] systemd[1]: Detected architecture x86-64. vm-test-run-nixbot-gitlab> gitlab # [ 6.562892] systemd[1]: Detected first boot. vm-test-run-nixbot-gitlab> gitlab # [ 6.567015] systemd[1]: Initializing machine ID from random generator. vm-test-run-nixbot-gitlab> gitlab # [ 6.717756] systemd[1]: bpf-restrict-fs: LSM BPF program attached vm-test-run-nixbot-gitlab> gitlab # [ 6.790449] systemd[1]: Applying preset policy. vm-test-run-nixbot-gitlab> gitlab # [ 6.969978] systemd[1]: Populated /etc with preset unit settings. vm-test-run-nixbot-gitlab> gitlab # [ 7.102919] systemd[1]: initrd-switch-root.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.105212] systemd[1]: Stopped initrd-switch-root.service. vm-test-run-nixbot-gitlab> gitlab # [ 7.108694] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. vm-test-run-nixbot-gitlab> gitlab # [ 7.112038] systemd[1]: Created slice Slice /system/getty. vm-test-run-nixbot-gitlab> gitlab # [ 7.114232] systemd[1]: Created slice GitLab DevOps Platform Slice. vm-test-run-nixbot-gitlab> gitlab # [ 7.116933] systemd[1]: Created slice User and Session Slice. vm-test-run-nixbot-gitlab> gitlab # [ 7.118547] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 7.120617] systemd[1]: Started Forward Password Requests to Wall Directory Watch. vm-test-run-nixbot-gitlab> gitlab # [ 7.122501] systemd[1]: Expecting device /dev/hvc0... vm-test-run-nixbot-gitlab> gitlab # [ 7.123801] systemd[1]: Expecting device /dev/ttyS0... vm-test-run-nixbot-gitlab> gitlab # [ 7.125150] systemd[1]: Reached target Local Encrypted Volumes. vm-test-run-nixbot-gitlab> gitlab # [ 7.126659] systemd[1]: Stopped target initrd-fs.target. vm-test-run-nixbot-gitlab> gitlab # [ 7.128009] systemd[1]: Stopped target initrd-root-fs.target. vm-test-run-nixbot-gitlab> gitlab # [ 7.129440] systemd[1]: Stopped target initrd-switch-root.target. vm-test-run-nixbot-gitlab> gitlab # [ 7.131034] systemd[1]: Reached target Virtual Machines and Containers. vm-test-run-nixbot-gitlab> gitlab # [ 7.132666] systemd[1]: Reached target Path Units. vm-test-run-nixbot-gitlab> gitlab # [ 7.133953] systemd[1]: Reached target Remote File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.135405] systemd[1]: Reached target Slice Units. vm-test-run-nixbot-gitlab> gitlab # [ 7.136729] systemd[1]: Reached target Swaps. vm-test-run-nixbot-gitlab> gitlab # [ 7.139076] systemd[1]: Listening on Process Core Dump Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.141263] systemd[1]: Listening on Credential Encryption/Decryption. vm-test-run-nixbot-gitlab> gitlab # [ 7.144854] systemd[1]: Starting Journal Log Access Socket... vm-test-run-nixbot-gitlab> gitlab # [ 7.147169] systemd[1]: Listening on Journal Audit Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.149051] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.151001] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.153612] systemd[1]: Listening on udev Control Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.157523] systemd[1]: Mounting Huge Pages File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.160673] systemd[1]: Mounting POSIX Message Queue File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.163852] systemd[1]: Mounting Kernel Debug File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.168792] systemd[1]: Mounting Kernel Trace File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.180980] systemd[1]: Starting Create List of Static Device Nodes... vm-test-run-nixbot-gitlab> gitlab # [ 7.182631] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio vm-test-run-nixbot-gitlab> gitlab # [ 7.187438] systemd[1]: Starting Load Kernel Module configfs... vm-test-run-nixbot-gitlab> gitlab # [ 7.191647] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm vm-test-run-nixbot-gitlab> gitlab # [ 7.198007] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.205921] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse vm-test-run-nixbot-gitlab> gitlab # [ 7.223746] systemd[1]: Mounting FUSE Control File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.254230] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 vm-test-run-nixbot-gitlab> gitlab # [ 7.268649] systemd[1]: Starting Journal Service... vm-test-run-nixbot-gitlab> gitlab # [ 7.271647] systemd[1]: Starting Load Kernel Modules... vm-test-run-nixbot-gitlab> gitlab # [ 7.276589] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... vm-test-run-nixbot-gitlab> gitlab # [ 7.287104] systemd[1]: Starting Remount Root and Kernel File Systems... vm-test-run-nixbot-gitlab> gitlab # [ 7.291754] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.297426] systemd-journald[786]: Collecting audit messages is enabled. vm-test-run-nixbot-gitlab> gitlab # [ 7.303805] systemd[1]: Starting Coldplug All udev Devices... vm-test-run-nixbot-gitlab> gitlab # [ 7.308161] loop: module loaded vm-test-run-nixbot-gitlab> gitlab # [ 7.317182] EXT4-fs (vda): re-mounted 0fcc4280-dcbd-4bd9-bcc9-80c7af117e8e. vm-test-run-nixbot-gitlab> gitlab # [ 7.054628] systemd[1]: Queued start job for default target Multi-User System. vm-test-run-nixbot-gitlab> gitlab # [ 7.318427] systemd[1]: Started Journal Service. vm-test-run-nixbot-gitlab> gitlab # [ 7.059664] systemd[1]: Unnecessary job was removed for /dev/vdb. vm-test-run-nixbot-gitlab> gitlab # [ 7.064139] systemd[1]: systemd-journald.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.067666] systemd-modules-load[787]: Using 3 probe threads vm-test-run-nixbot-gitlab> gitlab # [ 7.070431] systemd-modules-load[787]: Inserted module 'loop' vm-test-run-nixbot-gitlab> gitlab # [ 7.073594] systemd[1]: Listening on Journal Log Access Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.075465] systemd[1]: Mounted Huge Pages File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.077880] systemd-modules-load[787]: Inserted module 'tls' vm-test-run-nixbot-gitlab> gitlab # [ 7.079531] systemd[1]: Mounted POSIX Message Queue File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.081171] systemd[1]: Mounted Kernel Debug File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.082544] systemd[1]: Mounted Kernel Trace File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.083962] systemd[1]: Finished Create List of Static Device Nodes. vm-test-run-nixbot-gitlab> gitlab # [ 7.086265] systemd[1]: modprobe@configfs.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.088702] systemd[1]: Finished Load Kernel Module configfs. vm-test-run-nixbot-gitlab> gitlab # [ 7.090460] systemd-oomd[788]: No swap; memory pressure usage will be degraded vm-test-run-nixbot-gitlab> gitlab # [ 7.092885] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. vm-test-run-nixbot-gitlab> gitlab # [ 7.094451] systemd[1]: Mounted FUSE Control File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.096628] systemd[1]: Finished Load Kernel Modules. vm-test-run-nixbot-gitlab> gitlab # [ 7.098984] systemd[1]: Finished Remount Root and Kernel File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.112726] systemd[1]: Mounting Kernel Configuration File System... vm-test-run-nixbot-gitlab> gitlab # [ 7.115274] systemd[1]: Starting Firewall... vm-test-run-nixbot-gitlab> gitlab # [ 7.118207] systemd[1]: Starting Flush Journal to Persistent Storage... vm-test-run-nixbot-gitlab> gitlab # [ 7.120199] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.126785] systemd[1]: Starting Load/Save OS Random Seed... vm-test-run-nixbot-gitlab> gitlab # [ 7.130358] systemd[1]: Starting Apply Kernel Variables... vm-test-run-nixbot-gitlab> gitlab # [ 7.133363] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... vm-test-run-nixbot-gitlab> gitlab # [ 7.136393] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.402573] systemd-journald[786]: Received client request to flush runtime journal. vm-test-run-nixbot-gitlab> gitlab # [ 7.211056] systemd[1]: Mounted Kernel Configuration File System. vm-test-run-nixbot-gitlab> gitlab # [ 7.213284] systemd[1]: Finished Load/Save OS Random Seed. vm-test-run-nixbot-gitlab> gitlab # [ 7.214982] systemd[1]: Reached target First Boot Complete. vm-test-run-nixbot-gitlab> gitlab # [ 7.216707] systemd[1]: Finished Apply Kernel Variables. vm-test-run-nixbot-gitlab> gitlab # [ 7.218349] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. vm-test-run-nixbot-gitlab> gitlab # [ 7.220397] systemd[1]: Starting Create Static Device Nodes in /dev... vm-test-run-nixbot-gitlab> gitlab # [ 7.222060] systemd[1]: Finished Create Static Device Nodes in /dev. vm-test-run-nixbot-gitlab> gitlab # [ 7.223642] systemd[1]: Reached target Preparation for Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.225441] systemd[1]: Starting Rule-based Manager for Device Events and Files... vm-test-run-nixbot-gitlab> gitlab # [ 7.227405] systemd[1]: Finished Flush Journal to Persistent Storage. vm-test-run-nixbot-gitlab> gitlab # [ 7.236465] systemd-udevd[831]: Using default interface naming scheme 'v260'. vm-test-run-nixbot-gitlab> gitlab # [ 7.246955] systemd[1]: Finished Coldplug All udev Devices. vm-test-run-nixbot-gitlab> gitlab # [ 7.274236] systemd[1]: Started Rule-based Manager for Device Events and Files. vm-test-run-nixbot-gitlab> gitlab # [ 7.404193] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse vm-test-run-nixbot-gitlab> gitlab # [ 7.418639] systemd[1]: Finished Firewall. vm-test-run-nixbot-gitlab> gitlab # [ 7.428110] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 7.478457] (udev-worker)[903]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 7.480813] (udev-worker)[904]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. vm-test-run-nixbot-gitlab> gitlab # [ 7.484772] (udev-worker)[904]: Network interface NamePolicy= disabled on kernel command line. vm-test-run-nixbot-gitlab> gitlab # [ 7.504707] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 7.508293] systemd[1]: Condition check resulted in Virtio network device being skipped. vm-test-run-nixbot-gitlab> gitlab # [ 7.510677] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.513326] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 vm-test-run-nixbot-gitlab> gitlab # [ 7.517550] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore vm-test-run-nixbot-gitlab> gitlab # [ 7.520289] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.522510] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki vm-test-run-nixbot-gitlab> gitlab # [ 7.820597] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 vm-test-run-nixbot-gitlab> gitlab # [ 7.830817] ACPI: button: Power Button [PWRF] vm-test-run-nixbot-gitlab> gitlab # [ 7.837248] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 vm-test-run-nixbot-gitlab> gitlab # [ 7.850389] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD vm-test-run-nixbot-gitlab> gitlab # [ 7.858352] rtc_cmos 00:05: RTC can wake from S4 vm-test-run-nixbot-gitlab> gitlab # [ 7.861372] parport_pc 00:03: reported by Plug and Play ACPI vm-test-run-nixbot-gitlab> gitlab # [ 7.863110] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] vm-test-run-nixbot-gitlab> gitlab # [ 7.866826] mousedev: PS/2 mouse device common for all mice vm-test-run-nixbot-gitlab> gitlab # [ 7.877338] rtc_cmos 00:05: registered as rtc0 vm-test-run-nixbot-gitlab> gitlab # [ 7.878579] Floppy drive(s): fd0 is 2.88M AMI BIOS vm-test-run-nixbot-gitlab> gitlab # [ 7.880377] rtc_cmos 00:05: setting system clock to 2026-06-16T09:13:31 UTC (1781601211) vm-test-run-nixbot-gitlab> gitlab # [ 7.882910] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs vm-test-run-nixbot-gitlab> gitlab # [ 7.887440] systemd-journald[786]: Time jumped backwards, rotating. vm-test-run-nixbot-gitlab> gitlab # [ 7.889767] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console vm-test-run-nixbot-gitlab> gitlab # [ 7.892202] FDC 0 is a S82078B vm-test-run-nixbot-gitlab> gitlab # [ 7.895794] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4 vm-test-run-nixbot-gitlab> gitlab # [ 7.914052] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 vm-test-run-nixbot-gitlab> gitlab # [ 7.917666] Console: switching to colour dummy device 80x25 vm-test-run-nixbot-gitlab> gitlab # [ 7.922895] [drm] Found bochs VGA, ID 0xb0c5. vm-test-run-nixbot-gitlab> gitlab # [ 7.922897] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. vm-test-run-nixbot-gitlab> gitlab # [ 7.931530] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 vm-test-run-nixbot-gitlab> gitlab # [ 7.937028] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic vm-test-run-nixbot-gitlab> gitlab # [ 7.938411] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 vm-test-run-nixbot-gitlab> gitlab # [ 7.945118] fbcon: bochs-drmdrmfb (fb0) is primary device vm-test-run-nixbot-gitlab> gitlab # [ 7.951050] Console: switching to colour frame buffer device 160x50 vm-test-run-nixbot-gitlab> gitlab # [ 7.954230] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device vm-test-run-nixbot-gitlab> gitlab # [ 7.753135] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 8.042157] ppdev: user-space parallel port driver vm-test-run-nixbot-gitlab> gitlab # [ 7.786427] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.788774] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 7.795456] systemd[1]: Starting Virtual Console Setup... vm-test-run-nixbot-gitlab> gitlab # [ 8.079784] erofs (device vdb): failed to read inode meta block (nid: 109192949): -4 vm-test-run-nixbot-gitlab> gitlab # [ 7.819041] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.821482] systemd[1]: Stopped Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 8.090222] kvm_amd: TSC scaling supported vm-test-run-nixbot-gitlab> gitlab # [ 8.091640] kvm_amd: Nested Virtualization enabled vm-test-run-nixbot-gitlab> gitlab # [ 7.830790] systemd[1]: Start[ 8.094531] kvm_amd: Nested Paging enabled vm-test-run-nixbot-gitlab> gitlab # ing Virtual Cons[ 8.095890] kvm_amd: LBR virtualization supported vm-test-run-nixbot-gitlab> gitlab # ole Setup... vm-test-run-nixbot-gitlab> gitlab # [ 8.097071] kvm_amd: Virtual VMLOAD VMSAVE supported vm-test-run-nixbot-gitlab> gitlab # [ 8.098215] kvm_amd: Virtual GIF supported vm-test-run-nixbot-gitlab> gitlab # [ 8.099162] kvm_amd: Virtual NMI enabled vm-test-run-nixbot-gitlab> gitlab # [ 7.842495] systemd[1]: Mounting /run/wrappers... vm-test-run-nixbot-gitlab> gitlab # [ 7.857493] systemd[1]: Mounted /run/wrappers. vm-test-run-nixbot-gitlab> gitlab # [ 7.859387] systemd[1]: Reached target Local File Systems. vm-test-run-nixbot-gitlab> gitlab # [ 7.861333] systemd[1]: Listening on Boot Loader Control Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 7.863620] systemd[1]: Starting register-nix-paths.service... vm-test-run-nixbot-gitlab> gitlab # [ 7.866322] systemd[1]: Starting Create SUID/SGID Wrappers... vm-test-run-nixbot-gitlab> gitlab # [ 7.869545] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. vm-test-run-nixbot-gitlab> gitlab # [ 7.875307] systemd[1]: Starting Save Transient machine-id to Disk... vm-test-run-nixbot-gitlab> gitlab # [ 7.881104] systemd[1]: Starting Create System Files and Directories... vm-test-run-nixbot-gitlab> gitlab # [ 8.163303] EDAC MC: Ver: 3.0.0 vm-test-run-nixbot-gitlab> gitlab # [ 7.909991] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 7.914321] systemd[1]: Finished Save Transient machine-id to Disk. vm-test-run-nixbot-gitlab> gitlab # [ 7.920204] systemd-vconsole-setup[961]: Configuration of first virtual console was skipped, ignoring remaining ones. vm-test-run-nixbot-gitlab> gitlab # [ 7.923969] systemd[1]: Finished Virtual Console Setup. vm-test-run-nixbot-gitlab> gitlab # [ 7.932488] systemd[1]: Finished Create System Files and Directories. vm-test-run-nixbot-gitlab> gitlab # [ 7.936141] systemd[1]: Starting Rebuild Journal Catalog... vm-test-run-nixbot-gitlab> gitlab # [ 7.939297] systemd[1]: Starting Record System Boot/Shutdown in UTMP... vm-test-run-nixbot-gitlab> gitlab # [ 7.959363] systemd[1]: Finished Record System Boot/Shutdown in UTMP. vm-test-run-nixbot-gitlab> gitlab # [ 7.970278] systemd[1]: Finished Rebuild Journal Catalog. vm-test-run-nixbot-gitlab> gitlab # [ 7.972914] systemd[1]: Starting Update is Completed... vm-test-run-nixbot-gitlab> gitlab # [ 7.989813] systemd[1]: Finished Update is Completed. vm-test-run-nixbot-gitlab> gitlab # [ 8.036923] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.039934] systemd[1]: Finished Create SUID/SGID Wrappers. vm-test-run-nixbot-gitlab> gitlab # [ 8.041406] systemd[1]: suid-sgid-wrappers.service: Consumed 125ms CPU time over 170ms wall clock time, 4.6M memory peak, 1.1M read from disk. vm-test-run-nixbot-gitlab> gitlab # [ 8.275282] systemd[1]: Finished register-nix-paths.service. vm-test-run-nixbot-gitlab> gitlab # [ 8.277732] systemd[1]: Reached target System Initialization. vm-test-run-nixbot-gitlab> gitlab # [ 8.280171] systemd[1]: Started Discard unused filesystem blocks once a week. vm-test-run-nixbot-gitlab> gitlab # [ 8.282704] systemd[1]: Started logrotate.timer. vm-test-run-nixbot-gitlab> gitlab # [ 8.284746] systemd[1]: Started Daily Cleanup of Temporary Directories. vm-test-run-nixbot-gitlab> gitlab # [ 8.287344] systemd[1]: Reached target Timer Units. vm-test-run-nixbot-gitlab> gitlab # [ 8.289529] systemd[1]: Listening on D-Bus System Message Bus Socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.291878] systemd[1]: Listening on Nix Daemon Socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.299631] systemd[1]: Starting nixbot CI service socket... vm-test-run-nixbot-gitlab> gitlab # [ 8.302149] systemd[1]: Listening on Hostname Service Socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.304611] systemd[1]: Listening on nixbot CI service socket. vm-test-run-nixbot-gitlab> gitlab # [ 8.307212] systemd[1]: Reached target Socket Units. vm-test-run-nixbot-gitlab> gitlab # [ 8.309582] systemd[1]: Starting D-Bus System Message Bus... vm-test-run-nixbot-gitlab> gitlab # [ 8.377397] dbus-broker-launch[1051]: Looking up NSS user entry for 'systemd-timesync'... vm-test-run-nixbot-gitlab> gitlab # [ 8.380886] dbus-broker-launch[1051]: NSS returned no entry for 'systemd-timesync' vm-test-run-nixbot-gitlab> gitlab # [ 8.383849] dbus-broker-launch[1051]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" vm-test-run-nixbot-gitlab> gitlab # [ 8.388542] systemd[1]: Started D-Bus System Message Bus. vm-test-run-nixbot-gitlab> gitlab # [ 8.399202] dbus-broker-launch[1051]: Ready vm-test-run-nixbot-gitlab> gitlab # [ 8.403533] systemd[1]: Reached target Basic System. vm-test-run-nixbot-gitlab> gitlab # [ 8.407738] systemd[1]: Started backdoor.service. vm-test-run-nixbot-gitlab> gitlab # [ 8.411434] systemd[1]: Starting gitlab-config.service... vm-test-run-nixbot-gitlab> gitlab # [ 8.417395] systemd[1]: Starting Import lastlog data into lastlog2 database... vm-test-run-nixbot-gitlab> gitlab # [ 8.420641] systemd[1]: Starting Logrotate configuration check... vm-test-run-nixbot-gitlab> gitlab # [ 8.434502] systemd[1]: Starting Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 8.438345] logrotate[1056]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. vm-test-run-nixbot-gitlab> gitlab # [ 8.442262] logrotate[1056]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf vm-test-run-nixbot-gitlab> gitlab # [ 8.446300] logrotate[1056]: note: 'monthly' overrides previously specified 'weekly' vm-test-run-nixbot-gitlab> gitlab # [ 8.448479] logrotate[1056]: Reading state from file: /var/lib/logrotate.status vm-test-run-nixbot-gitlab> gitlab # [ 8.450410] logrotate[1056]: state file /var/lib/logrotate.status does not exist vm-test-run-nixbot-gitlab> gitlab # connecting to host... vm-test-run-nixbot-gitlab> gitlab # [ 8.455157] logrotate[1056]: Allocating hash table for state file, size 64 entries vm-test-run-nixbot-gitlab> gitlab # [ 8.457290] logrotate[1056]: Handling 4 logs vm-test-run-nixbot-gitlab> gitlab # [ 8.458930] logrotate[1056]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 8.463721] logrotate[1056]: considering log /var/log/btmp vm-test-run-nixbot-gitlab> gitlab # [ 8.465168] logrotate[1056]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 8.466499] logrotate[1056]: Now: 2026-06-16 09:13 vm-test-run-nixbot-gitlab> gitlab # [ 8.467986] logrotate[1056]: Last rotated at 2026-06-16 09:00 vm-test-run-nixbot-gitlab> gitlab # [ 8.469673] logrotate[1056]: log does not need rotating (log has already been rotated) vm-test-run-nixbot-gitlab> gitlab # [ 8.472123] logrotate[1056]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab: Guest shell says: b'Spawning backdoor root shell...\n' vm-test-run-nixbot-gitlab> gitlab: connected to guest root shell vm-test-run-nixbot-gitlab> gitlab: (connecting took 9.36 seconds) vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for the VM to finish booting, in 9.36 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 8.476233] logrotate[1056]: considering log /var/log/wtmp vm-test-run-nixbot-gitlab> gitlab # [ 8.477874] logrotate[1056]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 8.479041] logrotate[1056]: Now: 2026-06-16 09:13 vm-test-run-nixbot-gitlab> gitlab # [ 8.479976] logrotate[1056]: Last rotated at 2026-06-16 09:00 vm-test-run-nixbot-gitlab> gitlab # [ 8.481084] logrotate[1056]: log does not need rotating (log has already been rotated) vm-test-run-nixbot-gitlab> gitlab # [ 8.481743] logrotate[1056]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 8.485495] logrotate[1056]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1056) vm-test-run-nixbot-gitlab> gitlab # [ 8.485759] logrotate[1056]: considering log /var/gitlab/state/log/*.log vm-test-run-nixbot-gitlab> gitlab # [ 8.486873] logrotate[1056]: log /var/gitlab/state/log/*.log does not exist -- skipping vm-test-run-nixbot-gitlab> gitlab # [ 8.487834] logrotate[1056]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 8.492062] logrotate[1056]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1056) vm-test-run-nixbot-gitlab> gitlab # [ 8.493114] logrotate[1056]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed vm-test-run-nixbot-gitlab> gitlab # [ 8.493971] logrotate[1056]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1056) vm-test-run-nixbot-gitlab> gitlab # [ 8.497201] logrotate[1056]: considering log /var/log/nginx/*.log vm-test-run-nixbot-gitlab> gitlab # [ 8.498937] logrotate[1056]: log /var/log/nginx/*.log does not exist -- skipping vm-test-run-nixbot-gitlab> gitlab # [ 8.503151] logrotate[1056]: Creating new state vm-test-run-nixbot-gitlab> gitlab # [ 8.505404] logrotate[1056]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1056) vm-test-run-nixbot-gitlab> gitlab # [ 8.507214] systemd[1]: Starting Post-Boot Actions... vm-test-run-nixbot-gitlab> gitlab # [ 8.508784] systemd[1]: Started Reset console on configuration changes. vm-test-run-nixbot-gitlab> gitlab # [ 8.511102] systemd[1]: Starting resolvconf update... vm-test-run-nixbot-gitlab> gitlab # [ 8.512814] nsncd[1057]: Jun 16 09:13:32.377 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" vm-test-run-nixbot-gitlab> gitlab # [ 8.514331] systemd[1]: Finished Logrotate configuration check. vm-test-run-nixbot-gitlab> gitlab # [ 8.515532] systemd[1]: Finished Post-Boot Actions. vm-test-run-nixbot-gitlab> gitlab # [ 8.516674] systemd[1]: Finished Import lastlog data into lastlog2 database. vm-test-run-nixbot-gitlab> gitlab # [ 8.518084] systemd[1]: Started Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 8.519237] systemd[1]: Reached target Host and Network Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.520291] systemd[1]: Reached target User and Group Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.521129] systemd[1]: Starting User Login Management... vm-test-run-nixbot-gitlab> gitlab # [ 8.556891] systemd[1]: Stopped target Host and Network Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.558161] systemd[1]: Stopping Host and Network Name Lookups... vm-test-run-nixbot-gitlab> gitlab # [ 8.559467] systemd[1]: Stopped target User and Group Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.560445] systemd[1]: Stopping User and Group Name Lookups... vm-test-run-nixbot-gitlab> gitlab # [ 8.563289] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 8.565117] systemd[1]: nscd.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.566829] systemd[1]: Stopped Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 8.568559] systemd[1]: nscd.service: Consumed 24ms CPU time over 130ms wall clock time, 8.1M memory peak, 5.6M read from disk. vm-test-run-nixbot-gitlab> gitlab # [ 8.575106] systemd[1]: Starting Name Service Cache Daemon (nsncd)... vm-test-run-nixbot-gitlab> gitlab # [ 8.578612] systemd-logind[1100]: New seat seat0. vm-test-run-nixbot-gitlab> gitlab # [ 8.583129] systemd-logind[1100]: Watching system buttons on /dev/input/event2 (Power Button) vm-test-run-nixbot-gitlab> gitlab # [ 8.584374] systemd-logind[1100]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) vm-test-run-nixbot-gitlab> gitlab # [ 8.588456] systemd-logind[1100]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) vm-test-run-nixbot-gitlab> gitlab # [ 8.596939] systemd[1]: Started User Login Management. vm-test-run-nixbot-gitlab> gitlab # [ 8.601278] nsncd[1144]: Jun 16 09:13:32.483 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" vm-test-run-nixbot-gitlab> gitlab # [ 8.602542] systemd[1]: Starting linger-users.service... vm-test-run-nixbot-gitlab> gitlab # [ 8.603422] systemd[1]: Started Name Service Cache Daemon (nsncd). vm-test-run-nixbot-gitlab> gitlab # [ 8.604327] systemd[1]: Finished resolvconf update. vm-test-run-nixbot-gitlab> gitlab # [ 8.606269] systemd[1]: Reached target Preparation for Network. vm-test-run-nixbot-gitlab> gitlab # [ 8.607049] systemd[1]: Reached target Host and Network Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.607941] systemd[1]: Reached target User and Group Name Lookups. vm-test-run-nixbot-gitlab> gitlab # [ 8.630319] systemd[1]: Starting DHCP Client... vm-test-run-nixbot-gitlab> gitlab # [ 8.636142] systemd[1]: Starting Address configuration of eth1... vm-test-run-nixbot-gitlab> gitlab # [ 8.648836] systemd[1]: Starting Extra networking commands.... vm-test-run-nixbot-gitlab> gitlab # [ 8.653091] systemd[1]: linger-users.service: Deactivated successfully. vm-test-run-nixbot-gitlab> gitlab # [ 8.654275] systemd[1]: Finished linger-users.service. vm-test-run-nixbot-gitlab> gitlab # [ 8.674327] network-addresses-eth1-start[1169]: adding address 192.168.1.1/24... done vm-test-run-nixbot-gitlab> gitlab # [ 8.680865] network-addresses-eth1-start[1169]: adding address 2001:db8:1::1/64... done vm-test-run-nixbot-gitlab> gitlab # [ 8.688927] systemd[1]: Finished Address configuration of eth1. vm-test-run-nixbot-gitlab> gitlab # [ 8.732299] systemd[1]: Finished Extra networking commands.. vm-test-run-nixbot-gitlab> gitlab # [ 8.733897] systemd[1]: Reached target Network. vm-test-run-nixbot-gitlab> gitlab # [ 8.735704] systemd[1]: Starting gitlab-workhorse.service... vm-test-run-nixbot-gitlab> gitlab # [ 8.738424] dhcpcd[1183]: dhcpcd-10.3.2 starting vm-test-run-nixbot-gitlab> gitlab # [ 8.739806] systemd[1]: Starting Nginx Web Server... vm-test-run-nixbot-gitlab> gitlab # [ 8.745456] dhcpcd[1231]: dev: loaded udev vm-test-run-nixbot-gitlab> gitlab # [ 8.746781] systemd[1]: Starting PostgreSQL Server... vm-test-run-nixbot-gitlab> gitlab # [ 8.748211] systemd[1]: Starting Redis Server - redis-gitlab... vm-test-run-nixbot-gitlab> gitlab # [ 8.750874] systemd[1]: Starting Permit User Sessions... vm-test-run-nixbot-gitlab> gitlab # [ 9.016730] 8021q: 802.1Q VLAN Support v1.8 vm-test-run-nixbot-gitlab> gitlab # [ 9.017845] 8021q: adding VLAN 0 to HW filter on device eth1 vm-test-run-nixbot-gitlab> gitlab # [ 8.776226] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. vm-test-run-nixbot-gitlab> gitlab # [ 8.779131] systemd[1]: Finished Permit User Sessions. vm-test-run-nixbot-gitlab> gitlab # [ 8.784313] systemd[1]: Started Getty on tty1. vm-test-run-nixbot-gitlab> gitlab # [ 8.785649] systemd[1]: Reached target Login Prompts. vm-test-run-nixbot-gitlab> gitlab # [ 9.126743] cfg80211: Loading compiled-in X.509 certificates for regulatory database vm-test-run-nixbot-gitlab> gitlab # [ 9.132676] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' vm-test-run-nixbot-gitlab> gitlab # [ 9.134063] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' vm-test-run-nixbot-gitlab> gitlab # [ 9.137067] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 vm-test-run-nixbot-gitlab> gitlab # [ 9.138851] cfg80211: failed to load regulatory.db vm-test-run-nixbot-gitlab> gitlab # [ 8.889425] nginx-pre-start[1254]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok vm-test-run-nixbot-gitlab> gitlab # [ 8.893770] nginx-pre-start[1254]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful vm-test-run-nixbot-gitlab> gitlab # [ 8.897774] systemd[1]: Started Nginx Web Server. vm-test-run-nixbot-gitlab> gitlab # [ 9.175184] 8021q: adding VLAN 0 to HW filter on device eth0 vm-test-run-nixbot-gitlab> gitlab # [ 8.913804] dhcpcd[1231]: eth0: waiting for carrier vm-test-run-nixbot-gitlab> gitlab # [ 8.915041] dhcpcd[1231]: eth0: carrier acquired vm-test-run-nixbot-gitlab> gitlab # [ 8.918321] dhcpcd[1231]: DUID 00:01:00:01:31:c3:d0:3c:52:54:00:12:34:56 vm-test-run-nixbot-gitlab> gitlab # [ 8.920171] dhcpcd[1231]: eth0: IAID 00:12:34:56 vm-test-run-nixbot-gitlab> gitlab # [ 8.921586] dhcpcd[1231]: eth0: adding address fe80::5054:ff:fe12:3456 vm-test-run-nixbot-gitlab> gitlab # [ 8.942428] redis-server[1252]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. vm-test-run-nixbot-gitlab> gitlab # [ 8.944366] redis-server[1252]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo vm-test-run-nixbot-gitlab> gitlab # [ 8.945102] postgresql-pre-start[1262]: The files belonging to this database system will be owned by user "postgres". vm-test-run-nixbot-gitlab> gitlab # [ 8.946450] postgresql-pre-start[1262]: This user must also own the server process. vm-test-run-nixbot-gitlab> gitlab # [ 8.947953] redis-server[1252]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1252, just started vm-test-run-nixbot-gitlab> gitlab # [ 8.949542] postgresql-pre-start[1262]: The database cluster will be initialized with locale "en_US.UTF-8". vm-test-run-nixbot-gitlab> gitlab # [ 8.950763] postgresql-pre-start[1262]: The default database encoding has accordingly been set to "UTF8". vm-test-run-nixbot-gitlab> gitlab # [ 8.952075] postgresql-pre-start[1262]: The default text search configuration will be set to "english". vm-test-run-nixbot-gitlab> gitlab # [ 8.952768] postgresql-pre-start[1262]: Data page checksums are disabled. vm-test-run-nixbot-gitlab> gitlab # [ 8.954294] postgresql-pre-start[1262]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok vm-test-run-nixbot-gitlab> gitlab # [ 8.955197] postgresql-pre-start[1262]: creating subdirectories ... ok vm-test-run-nixbot-gitlab> gitlab # [ 8.956553] postgresql-pre-start[1262]: selecting dynamic shared memory implementation ... posix vm-test-run-nixbot-gitlab> gitlab # [ 8.958048] systemd[1]: Started Redis Server - redis-gitlab. vm-test-run-nixbot-gitlab> gitlab # [ 8.958988] redis-server[1252]: Configuration loaded vm-test-run-nixbot-gitlab> gitlab # [ 8.959920] redis[1252]: monotonic clock: POSIX clock_gettime vm-test-run-nixbot-gitlab> gitlab # [ 8.960959] redis[1252]: Running mode=standalone, port=0. vm-test-run-nixbot-gitlab> gitlab # [ 8.962361] redis[1252]: Server initialized vm-test-run-nixbot-gitlab> gitlab # [ 8.963321] redis[1252]: Ready to accept connections unix vm-test-run-nixbot-gitlab> gitlab # [ 8.964379] redis[1252]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. vm-test-run-nixbot-gitlab> gitlab # [ 8.981434] postgresql-pre-start[1262]: selecting default "max_connections" ... 100 vm-test-run-nixbot-gitlab> gitlab # [ 9.031289] postgresql-pre-start[1262]: selecting default "shared_buffers" ... 128MB vm-test-run-nixbot-gitlab> gitlab # [ 9.218147] systemd[1]: Started gitlab-workhorse.service. vm-test-run-nixbot-gitlab> gitlab # [ 9.404925] postgresql-pre-start[1262]: selecting default time zone ... UTC vm-test-run-nixbot-gitlab> gitlab # [ 9.407630] postgresql-pre-start[1262]: creating configuration files ... ok vm-test-run-nixbot-gitlab> gitlab # [ 9.497851] gitlab-workhorse[1288]: time="2026-06-16T09:13:33Z" level=info msg=Starting build_time=19700101.000000 version=18.11.4 vm-test-run-nixbot-gitlab> gitlab # [ 9.499145] gitlab-workhorse[1288]: time="2026-06-16T09:13:33Z" level=info msg="Using redis/go-redis" vm-test-run-nixbot-gitlab> gitlab # [ 9.500292] gitlab-workhorse[1288]: time="2026-06-16T09:13:33Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix vm-test-run-nixbot-gitlab> gitlab # [ 9.589323] postgresql-pre-start[1262]: running bootstrap script ... ok vm-test-run-nixbot-gitlab> gitlab # [ 10.087984] postgresql-pre-start[1262]: performing post-bootstrap initialization ... ok vm-test-run-nixbot-gitlab> gitlab # [ 10.137414] dhcpcd[1231]: eth0: soliciting an IPv6 router vm-test-run-nixbot-gitlab> gitlab # [ 10.139159] dhcpcd[1231]: eth0: Router Advertisement from fe80::2 vm-test-run-nixbot-gitlab> gitlab # [ 10.140484] dhcpcd[1231]: eth0: adding address fec0::5054:ff:fe12:3456/64 vm-test-run-nixbot-gitlab> gitlab # [ 10.141349] dhcpcd[1231]: eth0: adding route to fec0::/64 vm-test-run-nixbot-gitlab> gitlab # [ 10.142264] dhcpcd[1231]: eth0: adding default route via fe80::2 vm-test-run-nixbot-gitlab> gitlab # [ 10.545583] postgresql-pre-start[1262]: syncing data to disk ... ok vm-test-run-nixbot-gitlab> gitlab # [ 10.549098] postgresql-pre-start[1262]: initdb: warning: enabling "trust" authentication for local connections vm-test-run-nixbot-gitlab> gitlab # [ 10.552284] postgresql-pre-start[1262]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. vm-test-run-nixbot-gitlab> gitlab # [ 10.555169] postgresql-pre-start[1262]: Success. You can now start the database server using: vm-test-run-nixbot-gitlab> gitlab # [ 10.557965] postgresql-pre-start[1262]: pg_ctl -D /var/lib/postgresql/17 -l logfile start vm-test-run-nixbot-gitlab> gitlab # [ 10.692660] postgres[1313]: [1313] LOG: starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit vm-test-run-nixbot-gitlab> gitlab # [ 10.700280] postgres[1313]: [1313] LOG: listening on IPv6 address "::1", port 5432 vm-test-run-nixbot-gitlab> gitlab # [ 10.705462] postgres[1313]: [1313] LOG: listening on IPv4 address "127.0.0.1", port 5432 vm-test-run-nixbot-gitlab> gitlab # [ 10.710610] postgres[1313]: [1313] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" vm-test-run-nixbot-gitlab> gitlab # [ 10.716808] postgres[1329]: [1329] LOG: database system was shut down at 2026-06-16 09:13:33 GMT vm-test-run-nixbot-gitlab> gitlab # [ 10.722438] postgres[1313]: [1313] LOG: database system is ready to accept connections vm-test-run-nixbot-gitlab> gitlab # [ 10.730763] systemd[1]: Started PostgreSQL Server. vm-test-run-nixbot-gitlab> gitlab # [ 10.734197] systemd[1]: Starting PostgreSQL Setup Scripts... vm-test-run-nixbot-gitlab> gitlab # [ 10.778273] dhcpcd[1231]: eth0: soliciting a DHCP lease[ 11.047123] NET: Registered PF_PACKET protocol family vm-test-run-nixbot-gitlab> gitlab # vm-test-run-nixbot-gitlab> gitlab # [ 10.790977] dhcpcd[1231]: eth0: offered 10.0.2.15 from 10.0.2.2 vm-test-run-nixbot-gitlab> gitlab # [ 10.798812] dhcpcd[1231]: eth0: probing address 10.0.2.15/24 vm-test-run-nixbot-gitlab> gitlab # [ 10.894951] postgresql-setup-start[1345]: CREATE DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 10.920578] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1343]: mkdir -p /var/gitlab/state/home/.ssh: OK vm-test-run-nixbot-gitlab> gitlab # [ 10.923280] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1343]: chmod 700 /var/gitlab/state/home/.ssh: OK vm-test-run-nixbot-gitlab> gitlab # [ 10.944489] postgresql-setup-start[1356]: CREATE ROLE vm-test-run-nixbot-gitlab> gitlab # [ 10.956665] postgresql-setup-start[1362]: ALTER DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 10.971861] systemd[1]: Finished gitlab-config.service. vm-test-run-nixbot-gitlab> gitlab # [ 10.976446] systemd[1]: Started gitaly.service. vm-test-run-nixbot-gitlab> gitlab # [ 10.979540] postgresql-setup-start[1370]: CREATE ROLE vm-test-run-nixbot-gitlab> gitlab # [ 10.984151] systemd[1]: Finished PostgreSQL Setup Scripts. vm-test-run-nixbot-gitlab> gitlab # [ 10.986342] systemd[1]: Reached target PostgreSQL. vm-test-run-nixbot-gitlab> gitlab # [ 10.988589] systemd[1]: Starting gitlab-db-config.service... vm-test-run-nixbot-gitlab> gitlab # [ 10.990710] systemd[1]: Starting gitlab-postgresql.service... vm-test-run-nixbot-gitlab> gitlab # [ 11.087653] gitlab-postgresql-start[1383]: CREATE DATABASE vm-test-run-nixbot-gitlab> gitlab # [ 11.127527] gitlab-postgresql-start[1388]: CREATE EXTENSION vm-test-run-nixbot-gitlab> gitlab # [ 11.175278] gitlab-postgresql-start[1390]: CREATE EXTENSION vm-test-run-nixbot-gitlab> gitlab # [ 11.178822] systemd[1]: Finished gitlab-postgresql.service. vm-test-run-nixbot-gitlab> gitlab # [ 11.568108] gitaly[1373]: time="2026-06-16T09:13:35.450Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.569360] gitaly[1373]: time="2026-06-16T09:13:35.451Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.570427] gitaly[1373]: time="2026-06-16T09:13:35.451Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.575434] gitaly[1373]: time="2026-06-16T09:13:35.451Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.576207] gitaly[1373]: time="2026-06-16T09:13:35.451Z" level=info msg="Starting Gitaly" pid=1373 version=18.11.4 vm-test-run-nixbot-gitlab> gitlab # [ 11.785992] gitaly[1373]: time="2026-06-16T09:13:35.668Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=216 pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.787275] gitaly[1373]: time="2026-06-16T09:13:35.669Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.788142] gitaly[1373]: time="2026-06-16T09:13:35.669Z" level=info msg="finished initializing command factory" duration_ms=0 pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.789215] gitaly[1373]: time="2026-06-16T09:13:35.669Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1373/git-exec-3737690655.d/git pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.799364] gitaly[1373]: time="2026-06-16T09:13:35.682Z" level=info msg="finished detecting git version" duration_ms=12 pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.800848] gitaly[1373]: time="2026-06-16T09:13:35.682Z" level=info msg="using Git version" pid=1373 version=2.53.g vm-test-run-nixbot-gitlab> gitlab # [ 11.805748] gitaly[1373]: time="2026-06-16T09:13:35.688Z" level=info msg="clearing disk cache object folder" pid=1373 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 11.809077] gitaly[1373]: time="2026-06-16T09:13:35.692Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1075336981 pid=1373 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 11.810249] gitaly[1373]: time="2026-06-16T09:13:35.692Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1075336981 pid=1373 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 11.811426] gitaly[1373]: time="2026-06-16T09:13:35.692Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.812380] gitaly[1373]: time="2026-06-16T09:13:35.694Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1075336981 clear_duration_ms=0 pid=1373 storage=default vm-test-run-nixbot-gitlab> gitlab # [ 11.830872] gitaly[1373]: time="2026-06-16T09:13:35.713Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.902678] gitaly[1373]: time="2026-06-16T09:13:35.785Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.904143] gitaly[1373]: time="2026-06-16T09:13:35.787Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1373 schema=unix vm-test-run-nixbot-gitlab> gitlab # [ 11.905261] gitaly[1373]: time="2026-06-16T09:13:35.785Z" level=info msg="finished tempdir cleaner walk" pid=1373 storage=default time_ms=0 vm-test-run-nixbot-gitlab> gitlab # [ 11.906751] gitaly[1373]: time="2026-06-16T09:13:35.788Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1373/sock.d/intern pid=1373 schema=unix vm-test-run-nixbot-gitlab> gitlab # [ 11.907936] gitaly[1373]: time="2026-06-16T09:13:35.785Z" level=info msg="counting repositories" pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.909089] gitaly[1373]: time="2026-06-16T09:13:35.789Z" level=info msg="starting to count repositories" pid=1373 storage_path=/var/gitlab/state/repositories vm-test-run-nixbot-gitlab> gitlab # [ 11.909628] gitaly[1373]: time="2026-06-16T09:13:35.789Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1373 storage_path=/var/gitlab/state/repositories vm-test-run-nixbot-gitlab> gitlab # [ 11.910785] gitaly[1373]: time="2026-06-16T09:13:35.789Z" level=info msg="completed counting all repositories" counting_duration=0 pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.911729] gitaly[1373]: time="2026-06-16T09:13:35.791Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.912852] gitaly[1373]: time="2026-06-16T09:13:35.791Z" level=info msg="Started Gitaly" duration_ms=340 pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 11.914053] gitaly[1373]: time="2026-06-16T09:13:35.792Z" level=info msg="maintenance: daily scheduled" pid=1373 scheduled="2026-06-16 12:00:00 +0000 UTC" vm-test-run-nixbot-gitlab> gitlab # [ 12.421691] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1381]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 12.425966] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1381]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 12.446428] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1381]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 13.818319] gitaly[1373]: time="2026-06-16T09:13:37.700Z" level=info msg="License database preloaded" duration_ms=2248 pid=1373 vm-test-run-nixbot-gitlab> gitlab # [ 16.215768] dhcpcd[1231]: eth0: leased 10.0.2.15 for 86400 seconds vm-test-run-nixbot-gitlab> gitlab # [ 16.219653] dhcpcd[1231]: eth0: adding route to 10.0.2.0/24 vm-test-run-nixbot-gitlab> gitlab # [ 16.223281] dhcpcd[1231]: eth0: adding default route via 10.0.2.2 vm-test-run-nixbot-gitlab> gitlab # [ 16.364395] systemd[1]: Started DHCP Client. vm-test-run-nixbot-gitlab> gitlab # [ 16.369310] systemd[1]: Reached target Network is Online. vm-test-run-nixbot-gitlab> gitlab # [ 45.447893] postgres[1767]: [1767] ERROR: relation "feature_gates" does not exist at character 523 vm-test-run-nixbot-gitlab> gitlab # [ 45.452651] postgres[1767]: [1767] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), vm-test-run-nixbot-gitlab> gitlab # [ 45.456151] postgres[1767]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, vm-test-run-nixbot-gitlab> gitlab # [ 45.460333] postgres[1767]: c.collname, col_description(a.attrelid, a.attnum) AS comment, vm-test-run-nixbot-gitlab> gitlab # [ 45.465264] postgres[1767]: attidentity AS identity, vm-test-run-nixbot-gitlab> gitlab # [ 45.470134] postgres[1767]: attgenerated as attgenerated vm-test-run-nixbot-gitlab> gitlab # [ 45.473785] postgres[1767]: FROM pg_attribute a vm-test-run-nixbot-gitlab> gitlab # [ 45.477299] postgres[1767]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum vm-test-run-nixbot-gitlab> gitlab # [ 45.482199] postgres[1767]: LEFT JOIN pg_type t ON a.atttypid = t.oid vm-test-run-nixbot-gitlab> gitlab # [ 45.486319] postgres[1767]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation vm-test-run-nixbot-gitlab> gitlab # [ 45.490866] postgres[1767]: WHERE a.attrelid = '"feature_gates"'::regclass vm-test-run-nixbot-gitlab> gitlab # [ 45.495737] postgres[1767]: AND a.attnum > 0 AND NOT a.attisdropped vm-test-run-nixbot-gitlab> gitlab # [ 45.499422] postgres[1767]: ORDER BY a.attnum vm-test-run-nixbot-gitlab> gitlab # [ 45.502300] postgres[1767]: vm-test-run-nixbot-gitlab> gitlab # [ 45.677764] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1381]: W, [2026-06-16T09:14:09.559613 #1381] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 49.576181] gitlab-workhorse[1288]: time="2026-06-16T09:14:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 51.774753] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1831]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping vm-test-run-nixbot-gitlab> gitlab # [ 51.778995] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1831]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping vm-test-run-nixbot-gitlab> gitlab # [ 59.587574] gitlab-workhorse[1288]: time="2026-06-16T09:14:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 69.596982] gitlab-workhorse[1288]: time="2026-06-16T09:14:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 69.919145] postgres[1915]: [1915] ERROR: relation "table_without_models" does not exist vm-test-run-nixbot-gitlab> gitlab # [ 69.922532] postgres[1915]: [1915] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') vm-test-run-nixbot-gitlab> gitlab # [ 79.597965] gitlab-workhorse[1288]: time="2026-06-16T09:14:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 86.310259] systemd[1]: Finished gitlab-db-config.service. vm-test-run-nixbot-gitlab> gitlab # [ 86.318833] systemd[1]: Started gitlab-sidekiq.service. vm-test-run-nixbot-gitlab> gitlab # [ 86.326671] systemd[1]: Starting gitlab.service... vm-test-run-nixbot-gitlab> gitlab # [ 86.677372] bundle[2132]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 86.679408] bundle[2132]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 86.681457] bundle[2132]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 89.608569] gitlab-workhorse[1288]: time="2026-06-16T09:14:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 89.682772] sidekiq-cluster[2130]: {"severity":"INFO","time":"2026-06-16T09:14:53.547Z","message":"Starting cluster with 1 processes"} vm-test-run-nixbot-gitlab> gitlab # [ 89.790491] bundle[2132]: {"timestamp":"2026-06-16T09:14:53.650Z","pid":2132,"message":"* Enabling systemd notification integration"} vm-test-run-nixbot-gitlab> gitlab # [ 89.794980] bundle[2132]: {"timestamp":"2026-06-16T09:14:53.672Z","pid":2132,"message":"Puma starting in cluster mode..."} vm-test-run-nixbot-gitlab> gitlab # [ 89.797109] bundle[2132]: {"timestamp":"2026-06-16T09:14:53.672Z","pid":2132,"message":"* Puma version: 7.2.0 (\"On The Corner\")"} vm-test-run-nixbot-gitlab> gitlab # [ 89.800200] bundle[2132]: {"timestamp":"2026-06-16T09:14:53.672Z","pid":2132,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} vm-test-run-nixbot-gitlab> gitlab # [ 89.803362] bundle[2132]: {"timestamp":"2026-06-16T09:14:53.672Z","pid":2132,"message":"* Min threads: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 89.806177] bundle[2132]: {"timestamp":"2026-06-16T09:14:53.672Z","pid":2132,"message":"* Max threads: 4"} vm-test-run-nixbot-gitlab> gitlab # [ 89.809312] bundle[2132]: {"timestamp":"2026-06-16T09:14:53.672Z","pid":2132,"message":"* Environment: production"} vm-test-run-nixbot-gitlab> gitlab # [ 89.812179] bundle[2132]: {"timestamp":"2026-06-16T09:14:53.672Z","pid":2132,"message":"* Master PID: 2132"} vm-test-run-nixbot-gitlab> gitlab # [ 89.814941] bundle[2132]: {"timestamp":"2026-06-16T09:14:53.672Z","pid":2132,"message":"* Workers: 2"} vm-test-run-nixbot-gitlab> gitlab # [ 89.818357] bundle[2132]: {"timestamp":"2026-06-16T09:14:53.672Z","pid":2132,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} vm-test-run-nixbot-gitlab> gitlab # [ 89.821239] bundle[2132]: {"timestamp":"2026-06-16T09:14:53.672Z","pid":2132,"message":"* Preloading application"} vm-test-run-nixbot-gitlab> gitlab # [ 99.619278] gitlab-workhorse[1288]: time="2026-06-16T09:15:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 109.629862] gitlab-workhorse[1288]: time="2026-06-16T09:15:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 114.976816] bundle[2132]: W, [2026-06-16T09:15:18.858755 #2132] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 117.865434] sidekiq-cluster[2167]: 2026-06-16T09:15:21.747Z pid=2167 tid=u0v INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2167, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>430452736} vm-test-run-nixbot-gitlab> gitlab # [ 118.062852] sidekiq-cluster[2167]: W, [2026-06-16T09:15:21.945568 #2167] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 118.108463] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:21.991Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} vm-test-run-nixbot-gitlab> gitlab # [ 119.640731] gitlab-workhorse[1288]: time="2026-06-16T09:15:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 120.241374] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.123Z","message":"GitLab reliable fetch activated!"} vm-test-run-nixbot-gitlab> gitlab # [ 120.248350] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.123Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab # [ 120.310134] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.192Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.320398] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.199Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.329649] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.206Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.339729] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.212Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.348576] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.217Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.356105] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.221Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.362789] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.224Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.366654] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.227Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.370274] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.230Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.374062] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.234Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.378073] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.236Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.381858] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.239Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.385427] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.241Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.389275] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.244Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.393053] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.246Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.396798] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.249Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.400577] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.251Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.404168] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.254Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.407656] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.257Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.411447] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.260Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.415153] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.263Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.419208] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.266Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.423335] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.268Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.427071] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.271Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.430793] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.274Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.434590] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.276Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.438398] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.279Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.439777] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.282Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.440915] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.285Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.442105] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.288Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.442991] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.290Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.443803] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.293Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.444657] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.296Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.445824] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.299Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.447061] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.302Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.447849] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.304Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.449034] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.307Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.449875] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.309Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.481736] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.363Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.487842] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.370Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.493798] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.376Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.500142] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.383Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.507745] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.390Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.516266] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.399Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.520060] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.402Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.524930] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.407Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.533613] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.416Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.541152] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.423Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.553623] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.435Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.561730] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.443Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.566980] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.449Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.572834] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.455Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.576510] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.459Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.585968] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.468Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.590305] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.471Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.594343] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.475Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.598495] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.481Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.605349] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.488Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.611939] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.494Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.619925] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.502Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.626386] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.509Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.632792] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.515Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.638357] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.521Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.645198] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.528Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.651567] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.534Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.658312] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.541Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.667562] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.550Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.675498] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.558Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.683184] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.566Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.690875] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.573Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.699129] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.581Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.706931] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.589Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.715708] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.597Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.723840] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.606Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.732111] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.614Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.740140] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.622Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.748503] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.631Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.756562] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.639Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.765515] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.648Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.774730] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.657Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.783199] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.665Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.792424] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.675Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.800825] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.683Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.811788] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.694Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.820755] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.703Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.832607] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.715Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.840873] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.723Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.850632] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.733Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.858917] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.741Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.870791] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.753Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.879877] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.762Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.888735] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.771Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.897604] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.780Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.906977] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.789Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.916575] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.799Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.926481] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.809Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.939993] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.822Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.949369] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.832Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.958386] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.841Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.981672] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.864Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.989933] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.872Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 120.998953] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.881Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 121.009453] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.892Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 121.020823] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.902Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 121.031208] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.914Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 121.042142] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.924Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 121.051230] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.934Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 121.060166] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.942Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 121.071180] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.953Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 121.079976] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:24.962Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} vm-test-run-nixbot-gitlab> gitlab # [ 129.643566] gitlab-workhorse[1288]: time="2026-06-16T09:15:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 139.646809] gitlab-workhorse[1288]: time="2026-06-16T09:15:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= vm-test-run-nixbot-gitlab> gitlab # [ 144.011407] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:47.893Z","message":"Booted Rails 7.2.3 application in production environment"} vm-test-run-nixbot-gitlab> gitlab # [ 144.014231] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:47.894Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} vm-test-run-nixbot-gitlab> gitlab # [ 144.016187] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:47.894Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} vm-test-run-nixbot-gitlab> gitlab # [ 144.018840] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:47.894Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} vm-test-run-nixbot-gitlab> gitlab # [ 144.046118] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:47.928Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} vm-test-run-nixbot-gitlab> gitlab # [ 146.107288] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:49.989Z","message":"Cleaning working queues"} vm-test-run-nixbot-gitlab> gitlab # [ 146.116696] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:49.999Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c753bd5e2cf9e385e96a4dc1","created_at":"2026-06-16T09:14:49.517Z","trace_propagation_headers":{"sentry-trace":"fa0b81aa909c448a99fe1ac8a4f86100-471a85e396154ae2","baggage":"sentry-trace_id=fa0b81aa909c448a99fe1ac8a4f86100,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"6434ba6951a0ad3a9fa9a51ca95474d6","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-16T09:14:49.528Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c753bd5e2cf9e385e96a4dc1: start","job_status":"start","queue_duration_s":60.470541,"scheduling_latency_s":60.470651} vm-test-run-nixbot-gitlab> gitlab # [ 146.360361] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:15:50.242Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c753bd5e2cf9e385e96a4dc1","created_at":"2026-06-16T09:14:49.517Z","trace_propagation_headers":{"sentry-trace":"fa0b81aa909c448a99fe1ac8a4f86100-471a85e396154ae2","baggage":"sentry-trace_id=fa0b81aa909c448a99fe1ac8a4f86100,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"6434ba6951a0ad3a9fa9a51ca95474d6","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-16T09:14:49.528Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c753bd5e2cf9e385e96a4dc1: done: 0.244158 sec","job_status":"done","queue_duration_s":60.470541,"scheduling_latency_s":60.470651,"redis_calls":31,"redis_duration_s":0.009278,"redis_read_bytes":284,"redis_write_bytes":3897,"redis_feature_flag_calls":20,"redis_feature_flag_duration_s":0.00543,"redis_feature_flag_read_bytes":274,"redis_feature_flag_write_bytes":2650,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002408,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00144,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":16,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.096477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.092218703,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.244158,"completed_at":"2026-06-16T09:15:50.242Z","load_balancing_strategy":"primary","db_duration_s":0.056635,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 146.806634] bundle[2132]: {"timestamp":"2026-06-16T09:15:50.688Z","pid":2132,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} vm-test-run-nixbot-gitlab> gitlab # [ 146.809971] bundle[2132]: {"timestamp":"2026-06-16T09:15:50.688Z","pid":2132,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} vm-test-run-nixbot-gitlab> gitlab # [ 146.813547] bundle[2132]: {"timestamp":"2026-06-16T09:15:50.688Z","pid":2132,"message":"! #\u003cThread:0x0000703d851d9380 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} vm-test-run-nixbot-gitlab> gitlab # [ 146.816239] bundle[2132]: {"timestamp":"2026-06-16T09:15:50.688Z","pid":2132,"message":"! #\u003cThread:0x0000703d8540f348 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} vm-test-run-nixbot-gitlab> gitlab # [ 146.818948] bundle[2132]: {"timestamp":"2026-06-16T09:15:50.688Z","pid":2132,"message":"Use Ctrl-C to stop"} vm-test-run-nixbot-gitlab> gitlab # [ 146.861830] bundle[2132]: {"timestamp":"2026-06-16T09:15:50.689Z","pid":2132,"message":"Sending status to systemd every 1.0 sec"} vm-test-run-nixbot-gitlab> gitlab # [ 147.234542] bundle[2132]: {"timestamp":"2026-06-16T09:15:51.115Z","pid":2132,"message":"- Worker 0 (PID: 2712) booted in 0.34s, phase: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 147.287685] bundle[2132]: {"timestamp":"2026-06-16T09:15:51.170Z","pid":2132,"message":"- Worker 1 (PID: 2716) booted in 0.37s, phase: 0"} vm-test-run-nixbot-gitlab> gitlab # [ 147.295457] systemd[1]: Started gitlab.service. vm-test-run-nixbot-gitlab> gitlab # [ 147.298775] systemd[1]: Reached target Common target for all GitLab services.. vm-test-run-nixbot-gitlab> gitlab # [ 147.302536] systemd[1]: Starting setup-gitlab.service... vm-test-run-nixbot-gitlab> gitlab # [ 147.328304] setup-gitlab-start[2749]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' vm-test-run-nixbot-gitlab> gitlab # [ 150.067794] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:15:53 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 2709 vm-test-run-nixbot-gitlab> gitlab # [ 150.077455] setup-gitlab-start[2772]: ++ jq -r .access_token vm-test-run-nixbot-gitlab> gitlab # [ 150.079254] setup-gitlab-start[2771]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token vm-test-run-nixbot-gitlab> gitlab # [ 150.915597] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:15:54 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 816 vm-test-run-nixbot-gitlab> gitlab # [ 150.926844] setup-gitlab-start[2749]: + BEARER=0d59f68ec4dcbedcefb3add85736a9a13d57010605fcd88c0954a364817ec887 vm-test-run-nixbot-gitlab> gitlab # [ 150.930017] setup-gitlab-start[2749]: + AUTH='Authorization: Bearer 0d59f68ec4dcbedcefb3add85736a9a13d57010605fcd88c0954a364817ec887' vm-test-run-nixbot-gitlab> gitlab # [ 150.932870] setup-gitlab-start[2749]: + curl -fs -X PUT -H 'Authorization: Bearer 0d59f68ec4dcbedcefb3add85736a9a13d57010605fcd88c0954a364817ec887' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true' vm-test-run-nixbot-gitlab> gitlab # [ 151.727734] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:15:55 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 777 vm-test-run-nixbot-gitlab> gitlab # [ 151.731120] setup-gitlab-start[2749]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear vm-test-run-nixbot-gitlab> gitlab # [ 151.760855] runuser[2792]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) vm-test-run-nixbot-gitlab> gitlab # [ 152.137963] setup-gitlab-start[2793]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 152.141599] setup-gitlab-start[2793]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 152.144211] setup-gitlab-start[2793]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 152.755737] setup-gitlab-start[2793]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 152.759274] setup-gitlab-start[2793]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 152.761963] setup-gitlab-start[2793]: Source locally installed gems is ignoring # because it is missing extensions vm-test-run-nixbot-gitlab> gitlab # [ 158.077120] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:01.958Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"8851c15cc18c2c17e3ae5991","created_at":"2026-06-16T09:16:01.915Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f6ce598fd6bd1b15be4748be777febe8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:01.954Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-8851c15cc18c2c17e3ae5991: start","job_status":"start","queue_duration_s":0.004278,"scheduling_latency_s":0.004359} vm-test-run-nixbot-gitlab> gitlab # [ 159.142692] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:03.024Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"8851c15cc18c2c17e3ae5991","created_at":"2026-06-16T09:16:01.915Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f6ce598fd6bd1b15be4748be777febe8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:01.954Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-8851c15cc18c2c17e3ae5991: done: 1.066051 sec","job_status":"done","queue_duration_s":0.004278,"scheduling_latency_s":0.004359,"redis_calls":14,"redis_duration_s":0.018843,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.005995,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004432,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.008416,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.065,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":125,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.065,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.507853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.003163531,"exclusive_lock_hold_duration_s":0.925199991,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":1.066051,"completed_at":"2026-06-16T09:16:03.024Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.133563,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.252209] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:03.134Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"118885442eb5a0608f08a4ed","created_at":"2026-06-16T09:16:02.024Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"228cd3b26edb558d4733c6014b99ec19","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.050Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-118885442eb5a0608f08a4ed: start","job_status":"start","queue_duration_s":1.083261,"scheduling_latency_s":1.083464} vm-test-run-nixbot-gitlab> gitlab # [ 159.344530] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:03.226Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"118885442eb5a0608f08a4ed","created_at":"2026-06-16T09:16:02.024Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"228cd3b26edb558d4733c6014b99ec19","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.050Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-118885442eb5a0608f08a4ed: done: 0.092989 sec","job_status":"done","queue_duration_s":1.083261,"scheduling_latency_s":1.083464,"redis_calls":19,"redis_duration_s":0.010941000000000001,"redis_read_bytes":22,"redis_write_bytes":4773,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002438,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000803,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3003,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.006007,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001693,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000842286,"duration_s":0.092989,"completed_at":"2026-06-16T09:16:03.226Z","load_balancing_strategy":"primary","db_duration_s":0.004992,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.440230] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:03.322Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"03833d3f09fe181a1f545d83","created_at":"2026-06-16T09:16:02.069Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4c9e7797d0dfbeeafeac67fae154c56","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.092Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-03833d3f09fe181a1f545d83: start","job_status":"start","queue_duration_s":1.229091,"scheduling_latency_s":1.229171} vm-test-run-nixbot-gitlab> gitlab # [ 159.473419] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:03.356Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 159.490539] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:03.372Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"03833d3f09fe181a1f545d83","created_at":"2026-06-16T09:16:02.069Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4c9e7797d0dfbeeafeac67fae154c56","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.092Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-03833d3f09fe181a1f545d83: done: 0.050913 sec","job_status":"done","queue_duration_s":1.229091,"scheduling_latency_s":1.229171,"redis_calls":10,"redis_duration_s":0.006162,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001259,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003627,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001276,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001016889,"duration_s":0.050913,"completed_at":"2026-06-16T09:16:03.372Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001218,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.562111] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:03.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"890680eeb454f020df3a98ef","created_at":"2026-06-16T09:16:02.131Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81d42fa48d8cd145e7e6ee71a9b28504","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.145Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-890680eeb454f020df3a98ef: start","job_status":"start","queue_duration_s":1.298098,"scheduling_latency_s":1.298276} vm-test-run-nixbot-gitlab> gitlab # [ 159.621896] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:03.504Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"890680eeb454f020df3a98ef","created_at":"2026-06-16T09:16:02.131Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81d42fa48d8cd145e7e6ee71a9b28504","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.145Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-890680eeb454f020df3a98ef: done: 0.06008 sec","job_status":"done","queue_duration_s":1.298098,"scheduling_latency_s":1.298276,"redis_calls":12,"redis_duration_s":0.007514,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00247,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001302,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00102974,"duration_s":0.06008,"completed_at":"2026-06-16T09:16:03.504Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005899,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.627539] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:03.509Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"08b79d24803b1e2e87a77e24","created_at":"2026-06-16T09:16:02.154Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ded57212fa0a28a3a63b6e9ad5c6e5f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.165Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-08b79d24803b1e2e87a77e24: start","job_status":"start","queue_duration_s":1.344276,"scheduling_latency_s":1.344319} vm-test-run-nixbot-gitlab> gitlab # [ 159.766259] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:03.647Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"08b79d24803b1e2e87a77e24","created_at":"2026-06-16T09:16:02.154Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ded57212fa0a28a3a63b6e9ad5c6e5f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.165Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-08b79d24803b1e2e87a77e24: done: 0.137942 sec","job_status":"done","queue_duration_s":1.344276,"scheduling_latency_s":1.344319,"redis_calls":14,"redis_duration_s":0.011793,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001857,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006196,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00374,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000793397,"exclusive_lock_hold_duration_s":0.094594983,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.137942,"completed_at":"2026-06-16T09:16:03.647Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.075141,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.772582] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:03.652Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"a3907dce1a41d7902e454eca","created_at":"2026-06-16T09:16:02.184Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.191Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-a3907dce1a41d7902e454eca: start","job_status":"start","queue_duration_s":1.461084,"scheduling_latency_s":1.461128} vm-test-run-nixbot-gitlab> gitlab # [ 159.898762] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:03.781Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"a3907dce1a41d7902e454eca","created_at":"2026-06-16T09:16:02.184Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.191Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-a3907dce1a41d7902e454eca: done: 0.12861 sec","job_status":"done","queue_duration_s":1.461084,"scheduling_latency_s":1.461128,"redis_calls":71,"redis_duration_s":0.033396999999999996,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.004419,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001342,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.025445,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002191,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.083407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000614883,"duration_s":0.12861,"completed_at":"2026-06-16T09:16:03.781Z","load_balancing_strategy":"primary","db_duration_s":0.003431,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 159.969694] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:03.852Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"d2031e38929c7c796da47c01","created_at":"2026-06-16T09:16:02.209Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e54eebdc1812adfadef53dbd3491b614","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.216Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-d2031e38929c7c796da47c01: start","job_status":"start","queue_duration_s":1.636102,"scheduling_latency_s":1.636195} vm-test-run-nixbot-gitlab> gitlab # [ 160.008483] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:03.891Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"d2031e38929c7c796da47c01","created_at":"2026-06-16T09:16:02.209Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e54eebdc1812adfadef53dbd3491b614","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.216Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-d2031e38929c7c796da47c01: done: 0.038986 sec","job_status":"done","queue_duration_s":1.636102,"scheduling_latency_s":1.636195,"redis_calls":12,"redis_duration_s":0.00614,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001191,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002732,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002217,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001202388,"exclusive_lock_hold_duration_s":0.00943863,"duration_s":0.038986,"completed_at":"2026-06-16T09:16:03.890Z","load_balancing_strategy":"primary","db_duration_s":0.005738,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 160.077310] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:03.959Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"8fb152d4f0047a52b28e71dd","created_at":"2026-06-16T09:16:02.220Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea2ac2b361e86bedf4261e84835e4223","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.227Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-8fb152d4f0047a52b28e71dd: start","job_status":"start","queue_duration_s":1.731852,"scheduling_latency_s":1.731904} vm-test-run-nixbot-gitlab> gitlab # [ 160.130424] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:04.012Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 160.147329] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:04.028Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"8fb152d4f0047a52b28e71dd","created_at":"2026-06-16T09:16:02.220Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea2ac2b361e86bedf4261e84835e4223","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.227Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-8fb152d4f0047a52b28e71dd: done: 0.069228 sec","job_status":"done","queue_duration_s":1.731852,"scheduling_latency_s":1.731904,"redis_calls":10,"redis_duration_s":0.006103,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001233,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00371,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00116,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000956826,"duration_s":0.069228,"completed_at":"2026-06-16T09:16:04.028Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001319,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 160.217505] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:04.099Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8a71e8d6b0168741cc18a019","created_at":"2026-06-16T09:16:02.232Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0f9eaac8ed3e24de652ab139da5d97b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-16T09:16:02.239Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8a71e8d6b0168741cc18a019: start","job_status":"start","queue_duration_s":1.860284,"scheduling_latency_s":1.860357} vm-test-run-nixbot-gitlab> gitlab # [ 160.263232] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:04.145Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 160.282375] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:04.164Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8a71e8d6b0168741cc18a019","created_at":"2026-06-16T09:16:02.232Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0f9eaac8ed3e24de652ab139da5d97b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-16T09:16:02.239Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8a71e8d6b0168741cc18a019: done: 0.064963 sec","job_status":"done","queue_duration_s":1.860284,"scheduling_latency_s":1.860357,"redis_calls":10,"redis_duration_s":0.007745999999999999,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000736,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006212,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000798,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000723277,"duration_s":0.064963,"completed_at":"2026-06-16T09:16:04.164Z","load_balancing_strategy":"primary","db_duration_s":0.001479,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 160.352108] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:04.234Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"c08d34b17aa5072fce5155af","created_at":"2026-06-16T09:16:02.244Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c01d200ee981709ad727c914ca6f08c9","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.251Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-c08d34b17aa5072fce5155af: start","job_status":"start","queue_duration_s":1.982541,"scheduling_latency_s":1.982612} vm-test-run-nixbot-gitlab> gitlab # [ 160.421622] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:04.303Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"c08d34b17aa5072fce5155af","created_at":"2026-06-16T09:16:02.244Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c01d200ee981709ad727c914ca6f08c9","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.251Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-c08d34b17aa5072fce5155af: done: 0.069539 sec","job_status":"done","queue_duration_s":1.982541,"scheduling_latency_s":1.982612,"redis_calls":12,"redis_duration_s":0.005158,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.001072,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000823,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002455,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000808,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000767975,"duration_s":0.069539,"completed_at":"2026-06-16T09:16:04.303Z","load_balancing_strategy":"primary","db_duration_s":0.001363,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 160.493856] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:04.376Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"cc345cc48b9a11da474d051a","created_at":"2026-06-16T09:16:02.258Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a127ff74568bce33aa3d1c7933336184","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.264Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-cc345cc48b9a11da474d051a: start","job_status":"start","queue_duration_s":2.111444,"scheduling_latency_s":2.111517} vm-test-run-nixbot-gitlab> gitlab # [ 161.056064] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:04.938Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"cc345cc48b9a11da474d051a","created_at":"2026-06-16T09:16:02.258Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a127ff74568bce33aa3d1c7933336184","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.264Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-cc345cc48b9a11da474d051a: done: 0.562743 sec","job_status":"done","queue_duration_s":2.111444,"scheduling_latency_s":2.111517,"redis_calls":574,"redis_duration_s":0.208098,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000924,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002054,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.20512,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.395826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000448939,"duration_s":0.562743,"completed_at":"2026-06-16T09:16:04.938Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001276,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.164530] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.045Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"ad7b89065daac672a299c8d2","created_at":"2026-06-16T09:16:02.281Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.290Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-ad7b89065daac672a299c8d2: start","job_status":"start","queue_duration_s":2.755272,"scheduling_latency_s":2.755324} vm-test-run-nixbot-gitlab> gitlab # [ 161.283112] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.165Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"ad7b89065daac672a299c8d2","created_at":"2026-06-16T09:16:02.281Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.290Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-ad7b89065daac672a299c8d2: done: 0.119781 sec","job_status":"done","queue_duration_s":2.755272,"scheduling_latency_s":2.755324,"redis_calls":65,"redis_duration_s":0.031849,"redis_read_bytes":115,"redis_write_bytes":41332,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001951,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_calls":2,"redis_queues_duration_s":0.001282,"redis_queues_read_bytes":4,"redis_queues_write_bytes":37035,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.02717,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3374,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001446,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.083339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00072188,"duration_s":0.119781,"completed_at":"2026-06-16T09:16:05.165Z","load_balancing_strategy":"primary","db_duration_s":0.001962,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.383876] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7f73b7ee5a3c82f210061c79","created_at":"2026-06-16T09:16:02.309Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"290d7f75741bc41efa4cf9f83fbafeb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.320Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7f73b7ee5a3c82f210061c79: start","job_status":"start","queue_duration_s":2.945584,"scheduling_latency_s":2.945636} vm-test-run-nixbot-gitlab> gitlab # [ 161.484560] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.366Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7f73b7ee5a3c82f210061c79","created_at":"2026-06-16T09:16:02.309Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"290d7f75741bc41efa4cf9f83fbafeb5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.320Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7f73b7ee5a3c82f210061c79: done: 0.100633 sec","job_status":"done","queue_duration_s":2.945584,"scheduling_latency_s":2.945636,"redis_calls":12,"redis_duration_s":0.006985,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001201,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00314,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002644,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001671162,"exclusive_lock_hold_duration_s":0.022415697999999998,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.100633,"completed_at":"2026-06-16T09:16:05.366Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025833,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.489580] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.370Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"10d5f77728f830f022695485","created_at":"2026-06-16T09:16:02.332Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bdaf5eb3a16faed29c273e13ac44a55","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.343Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-10d5f77728f830f022695485: start","job_status":"start","queue_duration_s":3.027493,"scheduling_latency_s":3.027546} vm-test-run-nixbot-gitlab> gitlab # [ 161.526896] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.390Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 161.532997] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.396Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"10d5f77728f830f022695485","created_at":"2026-06-16T09:16:02.332Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bdaf5eb3a16faed29c273e13ac44a55","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.343Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-10d5f77728f830f022695485: done: 0.026401 sec","job_status":"done","queue_duration_s":3.027493,"scheduling_latency_s":3.027546,"redis_calls":10,"redis_duration_s":0.0036,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000596,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00226,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000744,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013537,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000362057,"duration_s":0.026401,"completed_at":"2026-06-16T09:16:05.396Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000554,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.610149] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.398Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"d9a9acff1bc59cd9b6dd4d22","created_at":"2026-06-16T09:16:02.363Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"ac69ffc40d119ea8118977a7a8f89b0f","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.373Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-d9a9acff1bc59cd9b6dd4d22: start","job_status":"start","queue_duration_s":3.0244,"scheduling_latency_s":3.024422} vm-test-run-nixbot-gitlab> gitlab # [ 161.679985] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.415Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"d9a9acff1bc59cd9b6dd4d22","created_at":"2026-06-16T09:16:02.363Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"ac69ffc40d119ea8118977a7a8f89b0f","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.373Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-d9a9acff1bc59cd9b6dd4d22: done: 0.016968 sec","job_status":"done","queue_duration_s":3.0244,"scheduling_latency_s":3.024422,"redis_calls":10,"redis_duration_s":0.002391,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000433,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001419,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000539,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000344178,"duration_s":0.016968,"completed_at":"2026-06-16T09:16:05.415Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000346,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.687270] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.416Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d2d64927d9314bef8d2830ae","created_at":"2026-06-16T09:16:02.388Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2911c1d70ab25e227ed6985cdf57673a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.402Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d2d64927d9314bef8d2830ae: start","job_status":"start","queue_duration_s":3.013882,"scheduling_latency_s":3.013904} vm-test-run-nixbot-gitlab> gitlab # [ 161.757100] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.436Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d2d64927d9314bef8d2830ae","created_at":"2026-06-16T09:16:02.388Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2911c1d70ab25e227ed6985cdf57673a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.402Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d2d64927d9314bef8d2830ae: done: 0.020153 sec","job_status":"done","queue_duration_s":3.013882,"scheduling_latency_s":3.013904,"redis_calls":10,"redis_duration_s":0.0024950000000000003,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00043,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001466,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000599,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000386921,"duration_s":0.020153,"completed_at":"2026-06-16T09:16:05.436Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001946,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.776185] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.438Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"0dc06716b46d53ee06ea1892","created_at":"2026-06-16T09:16:02.411Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"467801d6c80cb447476a682a24a1cd50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.424Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-0dc06716b46d53ee06ea1892: start","job_status":"start","queue_duration_s":3.013894,"scheduling_latency_s":3.013915} vm-test-run-nixbot-gitlab> gitlab # [ 161.846949] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.456Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"0dc06716b46d53ee06ea1892","created_at":"2026-06-16T09:16:02.411Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"467801d6c80cb447476a682a24a1cd50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.424Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-0dc06716b46d53ee06ea1892: done: 0.017659 sec","job_status":"done","queue_duration_s":3.013894,"scheduling_latency_s":3.013915,"redis_calls":10,"redis_duration_s":0.002584,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000434,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001635,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000515,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000399771,"duration_s":0.017659,"completed_at":"2026-06-16T09:16:05.456Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000372,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 161.920754] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.457Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"9cfb32dce3bada80719bff33","created_at":"2026-06-16T09:16:02.453Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b666928f7ce4f8ba5586e823eb000701","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.460Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-9cfb32dce3bada80719bff33: start","job_status":"start","queue_duration_s":2.997187,"scheduling_latency_s":2.997208} vm-test-run-nixbot-gitlab> gitlab # [ 161.921755] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.489Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"9cfb32dce3bada80719bff33","created_at":"2026-06-16T09:16:02.453Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b666928f7ce4f8ba5586e823eb000701","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.460Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-9cfb32dce3bada80719bff33: done: 0.031824 sec","job_status":"done","queue_duration_s":2.997187,"scheduling_latency_s":2.997208,"redis_calls":12,"redis_duration_s":0.006781,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001038,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003266,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002477,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000665448,"exclusive_lock_hold_duration_s":0.00579068,"duration_s":0.031824,"completed_at":"2026-06-16T09:16:05.489Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004358,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 162.003129] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.493Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d43ae67d0fc37049247b121f","created_at":"2026-06-16T09:16:02.475Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb4caba2bfbb84951c3a5f2f83cd9287","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-16T09:16:02.482Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d43ae67d0fc37049247b121f: start","job_status":"start","queue_duration_s":3.010984,"scheduling_latency_s":3.01106} vm-test-run-nixbot-gitlab> gitlab # [ 162.004512] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.517Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 162.006085] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.525Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d43ae67d0fc37049247b121f","created_at":"2026-06-16T09:16:02.475Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb4caba2bfbb84951c3a5f2f83cd9287","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-16T09:16:02.482Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d43ae67d0fc37049247b121f: done: 0.032908 sec","job_status":"done","queue_duration_s":3.010984,"scheduling_latency_s":3.01106,"redis_calls":10,"redis_duration_s":0.003,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000617,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001825,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000558,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000376863,"duration_s":0.032908,"completed_at":"2026-06-16T09:16:05.525Z","load_balancing_strategy":"primary","db_duration_s":0.000752,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 162.048833] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ba904c5d89f3dfe86159d314","created_at":"2026-06-16T09:16:02.487Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1b393e82d337cf59cf35166aaee9e50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.493Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ba904c5d89f3dfe86159d314: start","job_status":"start","queue_duration_s":3.033755,"scheduling_latency_s":3.033777} vm-test-run-nixbot-gitlab> gitlab # [ 162.126185] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.538Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 162.127860] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.544Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ba904c5d89f3dfe86159d314","created_at":"2026-06-16T09:16:02.487Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1b393e82d337cf59cf35166aaee9e50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.493Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ba904c5d89f3dfe86159d314: done: 0.017683 sec","job_status":"done","queue_duration_s":3.033755,"scheduling_latency_s":3.033777,"redis_calls":10,"redis_duration_s":0.002221,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000468,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001318,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000435,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000352279,"duration_s":0.017683,"completed_at":"2026-06-16T09:16:05.544Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000438,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 162.212045] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.546Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b78650d2505cc60d97df552c","created_at":"2026-06-16T09:16:02.498Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"342bb7f4b71143357dc766077bba1b36","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.506Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b78650d2505cc60d97df552c: start","job_status":"start","queue_duration_s":3.040249,"scheduling_latency_s":3.040271} vm-test-run-nixbot-gitlab> gitlab # [ 162.213548] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.569Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b78650d2505cc60d97df552c","created_at":"2026-06-16T09:16:02.498Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"342bb7f4b71143357dc766077bba1b36","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.506Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b78650d2505cc60d97df552c: done: 0.022551 sec","job_status":"done","queue_duration_s":3.040249,"scheduling_latency_s":3.040271,"redis_calls":12,"redis_duration_s":0.002874,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00047,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001331,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012732,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00045788,"exclusive_lock_hold_duration_s":0.00463299,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.022551,"completed_at":"2026-06-16T09:16:05.568Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002323,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 162.293163] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.570Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"8a1af1879496b2b458dbf3dc","created_at":"2026-06-16T09:16:02.513Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"30a34e48f99e5c65b1fd4bdb4ebb4f01","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.521Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-8a1af1879496b2b458dbf3dc: start","job_status":"start","queue_duration_s":3.049373,"scheduling_latency_s":3.049395} vm-test-run-nixbot-gitlab> gitlab # [ 162.294535] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.593Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"8a1af1879496b2b458dbf3dc","created_at":"2026-06-16T09:16:02.513Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"30a34e48f99e5c65b1fd4bdb4ebb4f01","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.521Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-8a1af1879496b2b458dbf3dc: done: 0.022901 sec","job_status":"done","queue_duration_s":3.049373,"scheduling_latency_s":3.049395,"redis_calls":10,"redis_duration_s":0.0023049999999999998,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000447,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001369,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000339428,"duration_s":0.022901,"completed_at":"2026-06-16T09:16:05.593Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002242,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 162.384406] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.594Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"3853de530d0a3e9c9d51b255","created_at":"2026-06-16T09:16:02.529Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a08c1eb7a00698227407dd73d5eb443e","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.537Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-3853de530d0a3e9c9d51b255: start","job_status":"start","queue_duration_s":3.057542,"scheduling_latency_s":3.057573} vm-test-run-nixbot-gitlab> gitlab # [ 162.385932] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.626Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"3853de530d0a3e9c9d51b255","created_at":"2026-06-16T09:16:02.529Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a08c1eb7a00698227407dd73d5eb443e","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.537Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-3853de530d0a3e9c9d51b255: done: 0.03132 sec","job_status":"done","queue_duration_s":3.057542,"scheduling_latency_s":3.057573,"redis_calls":10,"redis_duration_s":0.004226,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00048,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003083,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000663,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000371835,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.03132,"completed_at":"2026-06-16T09:16:05.626Z","load_balancing_strategy":"primary","db_duration_s":0.008523,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 162.461301] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.627Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0d19d28e4172b271398be514","created_at":"2026-06-16T09:16:02.543Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"007ef73f7ea3ceb17e8a83611e4b521b","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.550Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0d19d28e4172b271398be514: start","job_status":"start","queue_duration_s":3.07731,"scheduling_latency_s":3.077333} vm-test-run-nixbot-gitlab> gitlab # [ 162.462941] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.666Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0d19d28e4172b271398be514","created_at":"2026-06-16T09:16:02.543Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"007ef73f7ea3ceb17e8a83611e4b521b","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.550Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0d19d28e4172b271398be514: done: 0.038498 sec","job_status":"done","queue_duration_s":3.07731,"scheduling_latency_s":3.077333,"redis_calls":16,"redis_duration_s":0.008307,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.005365,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001972,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000378261,"duration_s":0.038498,"completed_at":"2026-06-16T09:16:05.666Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008709,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 162.561473] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.667Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"312670d5ca55176922dfc0ca","created_at":"2026-06-16T09:16:02.556Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6195eba04aeddc284eeea346699281c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-16T09:16:02.563Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-312670d5ca55176922dfc0ca: start","job_status":"start","queue_duration_s":3.104125,"scheduling_latency_s":3.104148} vm-test-run-nixbot-gitlab> gitlab # [ 162.563124] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.695Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"312670d5ca55176922dfc0ca","created_at":"2026-06-16T09:16:02.556Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6195eba04aeddc284eeea346699281c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-16T09:16:02.563Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-312670d5ca55176922dfc0ca: done: 0.027709 sec","job_status":"done","queue_duration_s":3.104125,"scheduling_latency_s":3.104148,"redis_calls":10,"redis_duration_s":0.002339,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000514,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000405079,"duration_s":0.027709,"completed_at":"2026-06-16T09:16:05.695Z","load_balancing_strategy":"primary","db_duration_s":0.002652,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 162.591309] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.696Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"a2bd691e37f4a333ab5dee84","created_at":"2026-06-16T09:16:02.575Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4f56228735f8caf093b32eab244bd6f","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.587Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-a2bd691e37f4a333ab5dee84: start","job_status":"start","queue_duration_s":3.109343,"scheduling_latency_s":3.109364} vm-test-run-nixbot-gitlab> gitlab # [ 162.660388] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.724Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"a2bd691e37f4a333ab5dee84","created_at":"2026-06-16T09:16:02.575Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4f56228735f8caf093b32eab244bd6f","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:02.587Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-a2bd691e37f4a333ab5dee84: done: 0.027339 sec","job_status":"done","queue_duration_s":3.109343,"scheduling_latency_s":3.109364,"redis_calls":13,"redis_duration_s":0.003118,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000489,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001608,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.008021410999999999,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.027339,"completed_at":"2026-06-16T09:16:05.724Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002971,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 162.741354] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.725Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b7827b823fd0dc29b634d637","created_at":"2026-06-16T09:16:03.191Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-83982733a6a546ba","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"228cd3b26edb558d4733c6014b99ec19","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.214Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b7827b823fd0dc29b634d637: start","job_status":"start","queue_duration_s":2.511381,"scheduling_latency_s":2.511404} vm-test-run-nixbot-gitlab> gitlab # [ 162.742897] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.743Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 162.744108] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.753Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b7827b823fd0dc29b634d637","created_at":"2026-06-16T09:16:03.191Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-83982733a6a546ba","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"228cd3b26edb558d4733c6014b99ec19","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.214Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b7827b823fd0dc29b634d637: done: 0.027857 sec","job_status":"done","queue_duration_s":2.511381,"scheduling_latency_s":2.511404,"redis_calls":13,"redis_duration_s":0.005653,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000479,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001607,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003567,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000341104,"duration_s":0.027857,"completed_at":"2026-06-16T09:16:05.753Z","load_balancing_strategy":"primary","db_duration_s":0.002874,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 162.769079] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.755Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ac2535256ef1a9d34d3ed5b7","created_at":"2026-06-16T09:16:03.191Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-83982733a6a546ba","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"228cd3b26edb558d4733c6014b99ec19","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.214Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ac2535256ef1a9d34d3ed5b7: start","job_status":"start","queue_duration_s":2.540728,"scheduling_latency_s":2.540749} vm-test-run-nixbot-gitlab> gitlab # [ 162.850393] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.765Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 162.851806] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.772Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ac2535256ef1a9d34d3ed5b7","created_at":"2026-06-16T09:16:03.191Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-83982733a6a546ba","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"228cd3b26edb558d4733c6014b99ec19","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.214Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ac2535256ef1a9d34d3ed5b7: done: 0.017121 sec","job_status":"done","queue_duration_s":2.540728,"scheduling_latency_s":2.540749,"redis_calls":10,"redis_duration_s":0.0025139999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001151,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001363,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010843,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017121,"completed_at":"2026-06-16T09:16:05.772Z","load_balancing_strategy":"primary","db_duration_s":0.001603,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 162.853167] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.773Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f0032859b720154f439c6842","created_at":"2026-06-16T09:16:03.191Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-83982733a6a546ba","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"228cd3b26edb558d4733c6014b99ec19","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.214Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f0032859b720154f439c6842: start","job_status":"start","queue_duration_s":2.559183,"scheduling_latency_s":2.559206} vm-test-run-nixbot-gitlab> gitlab # [ 162.932842] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.783Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 162.934183] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.789Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f0032859b720154f439c6842","created_at":"2026-06-16T09:16:03.191Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-83982733a6a546ba","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"228cd3b26edb558d4733c6014b99ec19","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.214Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f0032859b720154f439c6842: done: 0.01641 sec","job_status":"done","queue_duration_s":2.559183,"scheduling_latency_s":2.559206,"redis_calls":10,"redis_duration_s":0.002279,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000885,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001394,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01641,"completed_at":"2026-06-16T09:16:05.789Z","load_balancing_strategy":"primary","db_duration_s":0.001562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 162.935864] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.791Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7c4459223a3f389daae23140","created_at":"2026-06-16T09:16:03.191Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-83982733a6a546ba","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"228cd3b26edb558d4733c6014b99ec19","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.214Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7c4459223a3f389daae23140: start","job_status":"start","queue_duration_s":2.577101,"scheduling_latency_s":2.577124} vm-test-run-nixbot-gitlab> gitlab # [ 163.012297] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.801Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 163.013875] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.812Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7c4459223a3f389daae23140","created_at":"2026-06-16T09:16:03.191Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-83982733a6a546ba","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"228cd3b26edb558d4733c6014b99ec19","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.214Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7c4459223a3f389daae23140: done: 0.02153 sec","job_status":"done","queue_duration_s":2.577101,"scheduling_latency_s":2.577124,"redis_calls":10,"redis_duration_s":0.0038139999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00107,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002744,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02153,"completed_at":"2026-06-16T09:16:05.812Z","load_balancing_strategy":"primary","db_duration_s":0.001943,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 163.018261] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.814Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7582dcc7d3a6eeeafe9d459a","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7582dcc7d3a6eeeafe9d459a: start","job_status":"start","queue_duration_s":2.04424,"scheduling_latency_s":2.044271} vm-test-run-nixbot-gitlab> gitlab # [ 163.038852] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7582dcc7d3a6eeeafe9d459a","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7582dcc7d3a6eeeafe9d459a: done: 0.022843 sec","job_status":"done","queue_duration_s":2.04424,"scheduling_latency_s":2.044271,"redis_calls":16,"redis_duration_s":0.003905,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00052,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001235,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00215,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000384685,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022843,"completed_at":"2026-06-16T09:16:05.837Z","load_balancing_strategy":"primary","db_duration_s":0.000371,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 163.111241] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.838Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70fc267b5a9680d1be03b248","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70fc267b5a9680d1be03b248: start","job_status":"start","queue_duration_s":2.068524,"scheduling_latency_s":2.068545} vm-test-run-nixbot-gitlab> gitlab # [ 163.182593] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70fc267b5a9680d1be03b248","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70fc267b5a9680d1be03b248: done: 0.015688 sec","job_status":"done","queue_duration_s":2.068524,"scheduling_latency_s":2.068545,"redis_calls":13,"redis_duration_s":0.0029000000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015688,"completed_at":"2026-06-16T09:16:05.854Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 163.185229] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.855Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06c2b2a76fdab2a76b6eb036","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06c2b2a76fdab2a76b6eb036: start","job_status":"start","queue_duration_s":2.085681,"scheduling_latency_s":2.085704} vm-test-run-nixbot-gitlab> gitlab # [ 163.205337] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06c2b2a76fdab2a76b6eb036","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06c2b2a76fdab2a76b6eb036: done: 0.015711 sec","job_status":"done","queue_duration_s":2.085681,"scheduling_latency_s":2.085704,"redis_calls":13,"redis_duration_s":0.003274,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00237,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015711,"completed_at":"2026-06-16T09:16:05.871Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 163.285579] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85f7ac0b7603f399874b6c2b","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85f7ac0b7603f399874b6c2b: start","job_status":"start","queue_duration_s":2.102827,"scheduling_latency_s":2.102848} vm-test-run-nixbot-gitlab> gitlab # [ 163.287399] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85f7ac0b7603f399874b6c2b","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85f7ac0b7603f399874b6c2b: done: 0.018981 sec","job_status":"done","queue_duration_s":2.102827,"scheduling_latency_s":2.102848,"redis_calls":13,"redis_duration_s":0.006469,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002555,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018981,"completed_at":"2026-06-16T09:16:05.892Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 163.361235] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.895Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d082b8e1384bc524c0e17748","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d082b8e1384bc524c0e17748: start","job_status":"start","queue_duration_s":2.125429,"scheduling_latency_s":2.125452} vm-test-run-nixbot-gitlab> gitlab # [ 163.369226] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d082b8e1384bc524c0e17748","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d082b8e1384bc524c0e17748: done: 0.024444 sec","job_status":"done","queue_duration_s":2.125429,"scheduling_latency_s":2.125452,"redis_calls":13,"redis_duration_s":0.011526999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003363,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.008164,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024444,"completed_at":"2026-06-16T09:16:05.920Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 163.433942] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32995e1c3015a74fb226badc","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32995e1c3015a74fb226badc: start","job_status":"start","queue_duration_s":2.153432,"scheduling_latency_s":2.153454} vm-test-run-nixbot-gitlab> gitlab # [ 163.505348] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32995e1c3015a74fb226badc","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32995e1c3015a74fb226badc: done: 0.021618 sec","job_status":"done","queue_duration_s":2.153432,"scheduling_latency_s":2.153454,"redis_calls":13,"redis_duration_s":0.008366,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002233,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006133,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021618,"completed_at":"2026-06-16T09:16:05.945Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 163.515172] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0628d90508993f9998d8e737","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0628d90508993f9998d8e737: start","job_status":"start","queue_duration_s":2.176337,"scheduling_latency_s":2.176359} vm-test-run-nixbot-gitlab> gitlab # [ 163.591346] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0628d90508993f9998d8e737","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0628d90508993f9998d8e737: done: 0.015504 sec","job_status":"done","queue_duration_s":2.176337,"scheduling_latency_s":2.176359,"redis_calls":13,"redis_duration_s":0.0029430000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010386,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015504,"completed_at":"2026-06-16T09:16:05.962Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 163.597258] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d3e655110de2f8f7b6c37d5","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d3e655110de2f8f7b6c37d5: start","job_status":"start","queue_duration_s":2.193318,"scheduling_latency_s":2.19334} vm-test-run-nixbot-gitlab> gitlab # [ 163.676469] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d3e655110de2f8f7b6c37d5","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d3e655110de2f8f7b6c37d5: done: 0.015768 sec","job_status":"done","queue_duration_s":2.193318,"scheduling_latency_s":2.19334,"redis_calls":13,"redis_duration_s":0.0031609999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001181,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015768,"completed_at":"2026-06-16T09:16:05.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 163.746383] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1cea5b3b973d43cde9017f1","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1cea5b3b973d43cde9017f1: start","job_status":"start","queue_duration_s":2.210339,"scheduling_latency_s":2.210362} vm-test-run-nixbot-gitlab> gitlab # [ 163.747959] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1cea5b3b973d43cde9017f1","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1cea5b3b973d43cde9017f1: done: 0.015492 sec","job_status":"done","queue_duration_s":2.210339,"scheduling_latency_s":2.210362,"redis_calls":13,"redis_duration_s":0.003068,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002165,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015492,"completed_at":"2026-06-16T09:16:05.996Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 163.819431] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:05.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddea0299028caf093e79a53f","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddea0299028caf093e79a53f: start","job_status":"start","queue_duration_s":2.227293,"scheduling_latency_s":2.227315} vm-test-run-nixbot-gitlab> gitlab # [ 163.820836] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.014Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddea0299028caf093e79a53f","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddea0299028caf093e79a53f: done: 0.016556 sec","job_status":"done","queue_duration_s":2.227293,"scheduling_latency_s":2.227315,"redis_calls":13,"redis_duration_s":0.003918,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016556,"completed_at":"2026-06-16T09:16:06.014Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 163.842101] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14fd00e69692a1ea0bc988bc","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14fd00e69692a1ea0bc988bc: start","job_status":"start","queue_duration_s":2.2452,"scheduling_latency_s":2.245221} vm-test-run-nixbot-gitlab> gitlab # [ 163.862115] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14fd00e69692a1ea0bc988bc","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14fd00e69692a1ea0bc988bc: done: 0.015877 sec","job_status":"done","queue_duration_s":2.2452,"scheduling_latency_s":2.245221,"redis_calls":13,"redis_duration_s":0.003125,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002187,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015877,"completed_at":"2026-06-16T09:16:06.031Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 163.939253] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b4142d57f6baca1a7c077bf","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b4142d57f6baca1a7c077bf: start","job_status":"start","queue_duration_s":2.262427,"scheduling_latency_s":2.262449} vm-test-run-nixbot-gitlab> gitlab # [ 163.942474] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b4142d57f6baca1a7c077bf","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b4142d57f6baca1a7c077bf: done: 0.015501 sec","job_status":"done","queue_duration_s":2.262427,"scheduling_latency_s":2.262449,"redis_calls":13,"redis_duration_s":0.003083,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000913,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00217,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015501,"completed_at":"2026-06-16T09:16:06.048Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 164.023078] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.049Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ffcbdace3bea559c3548a70","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ffcbdace3bea559c3548a70: start","job_status":"start","queue_duration_s":2.27948,"scheduling_latency_s":2.279501} vm-test-run-nixbot-gitlab> gitlab # [ 164.024379] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ffcbdace3bea559c3548a70","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ffcbdace3bea559c3548a70: done: 0.015912 sec","job_status":"done","queue_duration_s":2.27948,"scheduling_latency_s":2.279501,"redis_calls":13,"redis_duration_s":0.003191,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002286,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015912,"completed_at":"2026-06-16T09:16:06.065Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 164.096312] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b7d0de393a916535ab2830c","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b7d0de393a916535ab2830c: start","job_status":"start","queue_duration_s":2.296891,"scheduling_latency_s":2.296957} vm-test-run-nixbot-gitlab> gitlab # [ 164.101234] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b7d0de393a916535ab2830c","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b7d0de393a916535ab2830c: done: 0.015994 sec","job_status":"done","queue_duration_s":2.296891,"scheduling_latency_s":2.296957,"redis_calls":13,"redis_duration_s":0.0030199999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010421,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015994,"completed_at":"2026-06-16T09:16:06.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 164.187286] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7f579940b927ba128e1332d","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7f579940b927ba128e1332d: start","job_status":"start","queue_duration_s":2.314348,"scheduling_latency_s":2.314369} vm-test-run-nixbot-gitlab> gitlab # [ 164.188888] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7f579940b927ba128e1332d","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7f579940b927ba128e1332d: done: 0.018272 sec","job_status":"done","queue_duration_s":2.314348,"scheduling_latency_s":2.314369,"redis_calls":13,"redis_duration_s":0.005853,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004887,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010333,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018272,"completed_at":"2026-06-16T09:16:06.102Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 164.260945] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5113ec972defbf761428c999","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5113ec972defbf761428c999: start","job_status":"start","queue_duration_s":2.334112,"scheduling_latency_s":2.334135} vm-test-run-nixbot-gitlab> gitlab # [ 164.262157] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5113ec972defbf761428c999","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5113ec972defbf761428c999: done: 0.016093 sec","job_status":"done","queue_duration_s":2.334112,"scheduling_latency_s":2.334135,"redis_calls":13,"redis_duration_s":0.0032500000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000989,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002261,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016093,"completed_at":"2026-06-16T09:16:06.120Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 164.334691] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.121Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5acc2423ca07c3fdc6cc501","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5acc2423ca07c3fdc6cc501: start","job_status":"start","queue_duration_s":2.351686,"scheduling_latency_s":2.351707} vm-test-run-nixbot-gitlab> gitlab # [ 164.340495] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5acc2423ca07c3fdc6cc501","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5acc2423ca07c3fdc6cc501: done: 0.016349 sec","job_status":"done","queue_duration_s":2.351686,"scheduling_latency_s":2.351707,"redis_calls":13,"redis_duration_s":0.003216,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001024,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002192,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016349,"completed_at":"2026-06-16T09:16:06.138Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 164.418378] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5b9a04d5fb4518bfd2a9f0f","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5b9a04d5fb4518bfd2a9f0f: start","job_status":"start","queue_duration_s":2.3693,"scheduling_latency_s":2.369322} vm-test-run-nixbot-gitlab> gitlab # [ 164.420123] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5b9a04d5fb4518bfd2a9f0f","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5b9a04d5fb4518bfd2a9f0f: done: 0.016006 sec","job_status":"done","queue_duration_s":2.3693,"scheduling_latency_s":2.369322,"redis_calls":13,"redis_duration_s":0.003164,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00228,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016006,"completed_at":"2026-06-16T09:16:06.155Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 164.482052] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aaab8c1038b5587f65e263ed","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aaab8c1038b5587f65e263ed: start","job_status":"start","queue_duration_s":2.386652,"scheduling_latency_s":2.386674} vm-test-run-nixbot-gitlab> gitlab # [ 164.555123] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aaab8c1038b5587f65e263ed","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aaab8c1038b5587f65e263ed: done: 0.016213 sec","job_status":"done","queue_duration_s":2.386652,"scheduling_latency_s":2.386674,"redis_calls":13,"redis_duration_s":0.0034990000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001048,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002451,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016213,"completed_at":"2026-06-16T09:16:06.173Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 164.561168] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.174Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"926f896e36b6fe027f887078","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-926f896e36b6fe027f887078: start","job_status":"start","queue_duration_s":2.404189,"scheduling_latency_s":2.404211} vm-test-run-nixbot-gitlab> gitlab # [ 164.581253] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"926f896e36b6fe027f887078","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-926f896e36b6fe027f887078: done: 0.016148 sec","job_status":"done","queue_duration_s":2.404189,"scheduling_latency_s":2.404211,"redis_calls":13,"redis_duration_s":0.0031590000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009909,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016148,"completed_at":"2026-06-16T09:16:06.190Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 164.643459] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70b48316f89f360446749153","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70b48316f89f360446749153: start","job_status":"start","queue_duration_s":2.4217,"scheduling_latency_s":2.421722} vm-test-run-nixbot-gitlab> gitlab # [ 164.712310] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"70b48316f89f360446749153","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-70b48316f89f360446749153: done: 0.015712 sec","job_status":"done","queue_duration_s":2.4217,"scheduling_latency_s":2.421722,"redis_calls":13,"redis_duration_s":0.003222,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015712,"completed_at":"2026-06-16T09:16:06.207Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 164.782259] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3b8ea2a9266059fb9758fb6","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3b8ea2a9266059fb9758fb6: start","job_status":"start","queue_duration_s":2.438786,"scheduling_latency_s":2.438808} vm-test-run-nixbot-gitlab> gitlab # [ 164.784239] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3b8ea2a9266059fb9758fb6","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3b8ea2a9266059fb9758fb6: done: 0.015244 sec","job_status":"done","queue_duration_s":2.438786,"scheduling_latency_s":2.438808,"redis_calls":13,"redis_duration_s":0.0030540000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002148,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010073,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015244,"completed_at":"2026-06-16T09:16:06.224Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 164.804473] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.225Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25bd6d889737bb944de7d1a4","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25bd6d889737bb944de7d1a4: start","job_status":"start","queue_duration_s":2.455442,"scheduling_latency_s":2.455463} vm-test-run-nixbot-gitlab> gitlab # [ 164.825165] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25bd6d889737bb944de7d1a4","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25bd6d889737bb944de7d1a4: done: 0.015115 sec","job_status":"done","queue_duration_s":2.455442,"scheduling_latency_s":2.455463,"redis_calls":13,"redis_duration_s":0.002807,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015115,"completed_at":"2026-06-16T09:16:06.240Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 164.901767] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32f533bba28fda49033cc4fd","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32f533bba28fda49033cc4fd: start","job_status":"start","queue_duration_s":2.471995,"scheduling_latency_s":2.472017} vm-test-run-nixbot-gitlab> gitlab # [ 164.904741] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.259Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32f533bba28fda49033cc4fd","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32f533bba28fda49033cc4fd: done: 0.017552 sec","job_status":"done","queue_duration_s":2.471995,"scheduling_latency_s":2.472017,"redis_calls":13,"redis_duration_s":0.005242,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000862,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00438,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017552,"completed_at":"2026-06-16T09:16:06.259Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 165.015305] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f62cde67cf560fbb543f3ed","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f62cde67cf560fbb543f3ed: start","job_status":"start","queue_duration_s":2.491124,"scheduling_latency_s":2.491146} vm-test-run-nixbot-gitlab> gitlab # [ 165.017510] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f62cde67cf560fbb543f3ed","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f62cde67cf560fbb543f3ed: done: 0.020318 sec","job_status":"done","queue_duration_s":2.491124,"scheduling_latency_s":2.491146,"redis_calls":13,"redis_duration_s":0.007766,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006858,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020318,"completed_at":"2026-06-16T09:16:06.281Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 165.051137] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1cb2e91f082193cbed680b3","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1cb2e91f082193cbed680b3: start","job_status":"start","queue_duration_s":2.512803,"scheduling_latency_s":2.512826} vm-test-run-nixbot-gitlab> gitlab # [ 165.070881] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1cb2e91f082193cbed680b3","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1cb2e91f082193cbed680b3: done: 0.014919 sec","job_status":"done","queue_duration_s":2.512803,"scheduling_latency_s":2.512826,"redis_calls":13,"redis_duration_s":0.002869,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014919,"completed_at":"2026-06-16T09:16:06.297Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 165.139307] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75e958dab30929394241a115","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75e958dab30929394241a115: start","job_status":"start","queue_duration_s":2.529074,"scheduling_latency_s":2.529094} vm-test-run-nixbot-gitlab> gitlab # [ 165.151263] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.314Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75e958dab30929394241a115","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75e958dab30929394241a115: done: 0.015043 sec","job_status":"done","queue_duration_s":2.529074,"scheduling_latency_s":2.529094,"redis_calls":13,"redis_duration_s":0.002936,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010172,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015043,"completed_at":"2026-06-16T09:16:06.314Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 165.264158] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8ef441cbfc640a8b03ad890","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8ef441cbfc640a8b03ad890: start","job_status":"start","queue_duration_s":2.545532,"scheduling_latency_s":2.545553} vm-test-run-nixbot-gitlab> gitlab # [ 165.265550] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8ef441cbfc640a8b03ad890","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8ef441cbfc640a8b03ad890: done: 0.015207 sec","job_status":"done","queue_duration_s":2.545532,"scheduling_latency_s":2.545553,"redis_calls":13,"redis_duration_s":0.0029909999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001077,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015207,"completed_at":"2026-06-16T09:16:06.330Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 165.295356] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8eee32741a0949ff22c02c88","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8eee32741a0949ff22c02c88: start","job_status":"start","queue_duration_s":2.562084,"scheduling_latency_s":2.562105} vm-test-run-nixbot-gitlab> gitlab # [ 165.314903] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8eee32741a0949ff22c02c88","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8eee32741a0949ff22c02c88: done: 0.021216 sec","job_status":"done","queue_duration_s":2.562084,"scheduling_latency_s":2.562105,"redis_calls":13,"redis_duration_s":0.008384,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002108,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006276,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021216,"completed_at":"2026-06-16T09:16:06.353Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 165.400464] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b230bde4b9dd0bfbc3b27d12","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b230bde4b9dd0bfbc3b27d12: start","job_status":"start","queue_duration_s":2.586262,"scheduling_latency_s":2.586285} vm-test-run-nixbot-gitlab> gitlab # [ 165.402835] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.547Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b230bde4b9dd0bfbc3b27d12","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b230bde4b9dd0bfbc3b27d12: done: 0.190554 sec","job_status":"done","queue_duration_s":2.586262,"scheduling_latency_s":2.586285,"redis_calls":13,"redis_duration_s":0.011074,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003459,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007615,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012255,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.190554,"completed_at":"2026-06-16T09:16:06.547Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 165.510893] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42dd7677284f8f038568d3ed","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42dd7677284f8f038568d3ed: start","job_status":"start","queue_duration_s":2.778431,"scheduling_latency_s":2.778459} vm-test-run-nixbot-gitlab> gitlab # [ 165.512297] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42dd7677284f8f038568d3ed","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42dd7677284f8f038568d3ed: done: 0.016486 sec","job_status":"done","queue_duration_s":2.778431,"scheduling_latency_s":2.778459,"redis_calls":13,"redis_duration_s":0.002969,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000944,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016486,"completed_at":"2026-06-16T09:16:06.565Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 165.587819] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.566Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6028a67cd3fb7926db1bea36","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6028a67cd3fb7926db1bea36: start","job_status":"start","queue_duration_s":2.79623,"scheduling_latency_s":2.796251} vm-test-run-nixbot-gitlab> gitlab # [ 165.589334] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6028a67cd3fb7926db1bea36","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6028a67cd3fb7926db1bea36: done: 0.015959 sec","job_status":"done","queue_duration_s":2.79623,"scheduling_latency_s":2.796251,"redis_calls":13,"redis_duration_s":0.0033510000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002403,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015959,"completed_at":"2026-06-16T09:16:06.582Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 165.664527] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a2d0dee8b71058012e8a312","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a2d0dee8b71058012e8a312: start","job_status":"start","queue_duration_s":2.813645,"scheduling_latency_s":2.813667} vm-test-run-nixbot-gitlab> gitlab # [ 165.666142] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a2d0dee8b71058012e8a312","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a2d0dee8b71058012e8a312: done: 0.015148 sec","job_status":"done","queue_duration_s":2.813645,"scheduling_latency_s":2.813667,"redis_calls":13,"redis_duration_s":0.002883,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000902,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015148,"completed_at":"2026-06-16T09:16:06.598Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 165.695933] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5676c06ebc3e7ce3e9f4603","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5676c06ebc3e7ce3e9f4603: start","job_status":"start","queue_duration_s":2.830234,"scheduling_latency_s":2.830256} vm-test-run-nixbot-gitlab> gitlab # [ 165.781633] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5676c06ebc3e7ce3e9f4603","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5676c06ebc3e7ce3e9f4603: done: 0.015251 sec","job_status":"done","queue_duration_s":2.830234,"scheduling_latency_s":2.830256,"redis_calls":13,"redis_duration_s":0.002843,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010218,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015251,"completed_at":"2026-06-16T09:16:06.615Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 165.783903] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c25e5decac66e963ed5e293b","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c25e5decac66e963ed5e293b: start","job_status":"start","queue_duration_s":2.847026,"scheduling_latency_s":2.847047} vm-test-run-nixbot-gitlab> gitlab # [ 165.857376] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c25e5decac66e963ed5e293b","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c25e5decac66e963ed5e293b: done: 0.015608 sec","job_status":"done","queue_duration_s":2.847026,"scheduling_latency_s":2.847047,"redis_calls":13,"redis_duration_s":0.0033290000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00238,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015608,"completed_at":"2026-06-16T09:16:06.632Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 165.859150] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa969af57f61fa3523eb3e46","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa969af57f61fa3523eb3e46: start","job_status":"start","queue_duration_s":2.864278,"scheduling_latency_s":2.864299} vm-test-run-nixbot-gitlab> gitlab # [ 165.928102] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa969af57f61fa3523eb3e46","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa969af57f61fa3523eb3e46: done: 0.01526 sec","job_status":"done","queue_duration_s":2.864278,"scheduling_latency_s":2.864299,"redis_calls":13,"redis_duration_s":0.003002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010391,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01526,"completed_at":"2026-06-16T09:16:06.649Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 165.935801] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.651Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"caebf9cfc0bdd947d601ce90","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-caebf9cfc0bdd947d601ce90: start","job_status":"start","queue_duration_s":2.880978,"scheduling_latency_s":2.881} vm-test-run-nixbot-gitlab> gitlab # [ 166.003152] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"caebf9cfc0bdd947d601ce90","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-caebf9cfc0bdd947d601ce90: done: 0.071087 sec","job_status":"done","queue_duration_s":2.880978,"scheduling_latency_s":2.881,"redis_calls":13,"redis_duration_s":0.003144,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001201,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.063985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.071087,"completed_at":"2026-06-16T09:16:06.722Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 166.076972] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7493c7ea978e24f912039b9b","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7493c7ea978e24f912039b9b: start","job_status":"start","queue_duration_s":2.953626,"scheduling_latency_s":2.953654} vm-test-run-nixbot-gitlab> gitlab # [ 166.078319] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7493c7ea978e24f912039b9b","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7493c7ea978e24f912039b9b: done: 0.017897 sec","job_status":"done","queue_duration_s":2.953626,"scheduling_latency_s":2.953654,"redis_calls":13,"redis_duration_s":0.0052829999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.003252,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017897,"completed_at":"2026-06-16T09:16:06.741Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 166.148200] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf87031c7a49c62e3f05f814","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf87031c7a49c62e3f05f814: start","job_status":"start","queue_duration_s":2.972846,"scheduling_latency_s":2.972868} vm-test-run-nixbot-gitlab> gitlab # [ 166.149893] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf87031c7a49c62e3f05f814","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf87031c7a49c62e3f05f814: done: 0.015265 sec","job_status":"done","queue_duration_s":2.972846,"scheduling_latency_s":2.972868,"redis_calls":13,"redis_duration_s":0.0029820000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000988,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001994,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015265,"completed_at":"2026-06-16T09:16:06.758Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 166.219828] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7eb98ceb71c56fb141a2a8a","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7eb98ceb71c56fb141a2a8a: start","job_status":"start","queue_duration_s":2.989428,"scheduling_latency_s":2.989449} vm-test-run-nixbot-gitlab> gitlab # [ 166.221223] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7eb98ceb71c56fb141a2a8a","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7eb98ceb71c56fb141a2a8a: done: 0.01491 sec","job_status":"done","queue_duration_s":2.989428,"scheduling_latency_s":2.989449,"redis_calls":13,"redis_duration_s":0.002961,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01491,"completed_at":"2026-06-16T09:16:06.774Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 166.290916] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8824079f000a2cc7c96c16f","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8824079f000a2cc7c96c16f: start","job_status":"start","queue_duration_s":3.0056,"scheduling_latency_s":3.00563} vm-test-run-nixbot-gitlab> gitlab # [ 166.292277] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8824079f000a2cc7c96c16f","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8824079f000a2cc7c96c16f: done: 0.014864 sec","job_status":"done","queue_duration_s":3.0056,"scheduling_latency_s":3.00563,"redis_calls":13,"redis_duration_s":0.002795,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014864,"completed_at":"2026-06-16T09:16:06.790Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 166.334401] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.792Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85e55874a72063d1ee4796a1","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85e55874a72063d1ee4796a1: start","job_status":"start","queue_duration_s":3.022516,"scheduling_latency_s":3.022561} vm-test-run-nixbot-gitlab> gitlab # [ 166.404529] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.810Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85e55874a72063d1ee4796a1","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85e55874a72063d1ee4796a1: done: 0.018066 sec","job_status":"done","queue_duration_s":3.022516,"scheduling_latency_s":3.022561,"redis_calls":13,"redis_duration_s":0.002994,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018066,"completed_at":"2026-06-16T09:16:06.810Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 166.413433] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0924e47b41bfe598e497915","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0924e47b41bfe598e497915: start","job_status":"start","queue_duration_s":3.041812,"scheduling_latency_s":3.041833} vm-test-run-nixbot-gitlab> gitlab # [ 166.483196] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0924e47b41bfe598e497915","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0924e47b41bfe598e497915: done: 0.015861 sec","job_status":"done","queue_duration_s":3.041812,"scheduling_latency_s":3.041833,"redis_calls":13,"redis_duration_s":0.003212,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015861,"completed_at":"2026-06-16T09:16:06.827Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 166.494273] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7928ebef4f6ce4067bad466d","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7928ebef4f6ce4067bad466d: start","job_status":"start","queue_duration_s":3.05902,"scheduling_latency_s":3.059043} vm-test-run-nixbot-gitlab> gitlab # [ 166.575352] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7928ebef4f6ce4067bad466d","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7928ebef4f6ce4067bad466d: done: 0.014983 sec","job_status":"done","queue_duration_s":3.05902,"scheduling_latency_s":3.059043,"redis_calls":13,"redis_duration_s":0.002983,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014983,"completed_at":"2026-06-16T09:16:06.844Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 166.577189] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76384fedd27e8c54b9f4bf71","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76384fedd27e8c54b9f4bf71: start","job_status":"start","queue_duration_s":3.075278,"scheduling_latency_s":3.075298} vm-test-run-nixbot-gitlab> gitlab # [ 166.595865] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.860Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76384fedd27e8c54b9f4bf71","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76384fedd27e8c54b9f4bf71: done: 0.014936 sec","job_status":"done","queue_duration_s":3.075278,"scheduling_latency_s":3.075298,"redis_calls":13,"redis_duration_s":0.002851,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014936,"completed_at":"2026-06-16T09:16:06.860Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 166.693210] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32345d516b3bdbb01f843ba2","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32345d516b3bdbb01f843ba2: start","job_status":"start","queue_duration_s":3.091494,"scheduling_latency_s":3.091517} vm-test-run-nixbot-gitlab> gitlab # [ 166.694892] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32345d516b3bdbb01f843ba2","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32345d516b3bdbb01f843ba2: done: 0.014914 sec","job_status":"done","queue_duration_s":3.091494,"scheduling_latency_s":3.091517,"redis_calls":13,"redis_duration_s":0.00281,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014914,"completed_at":"2026-06-16T09:16:06.876Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 166.739986] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c940e2354c2aa7098d17a42","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c940e2354c2aa7098d17a42: start","job_status":"start","queue_duration_s":3.108093,"scheduling_latency_s":3.108114} vm-test-run-nixbot-gitlab> gitlab # [ 166.811370] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c940e2354c2aa7098d17a42","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c940e2354c2aa7098d17a42: done: 0.014932 sec","job_status":"done","queue_duration_s":3.108093,"scheduling_latency_s":3.108114,"redis_calls":13,"redis_duration_s":0.003209,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002339,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014932,"completed_at":"2026-06-16T09:16:06.893Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 166.820910] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"351deff9a43eb737d4988512","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-351deff9a43eb737d4988512: start","job_status":"start","queue_duration_s":3.124289,"scheduling_latency_s":3.124312} vm-test-run-nixbot-gitlab> gitlab # [ 166.841045] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.909Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"351deff9a43eb737d4988512","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-351deff9a43eb737d4988512: done: 0.015022 sec","job_status":"done","queue_duration_s":3.124289,"scheduling_latency_s":3.124312,"redis_calls":13,"redis_duration_s":0.0028079999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015022,"completed_at":"2026-06-16T09:16:06.909Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 166.904449] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7feffb20bcde6ca9085b0de","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7feffb20bcde6ca9085b0de: start","job_status":"start","queue_duration_s":3.140586,"scheduling_latency_s":3.140607} vm-test-run-nixbot-gitlab> gitlab # [ 166.972464] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7feffb20bcde6ca9085b0de","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7feffb20bcde6ca9085b0de: done: 0.014998 sec","job_status":"done","queue_duration_s":3.140586,"scheduling_latency_s":3.140607,"redis_calls":13,"redis_duration_s":0.002848,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014998,"completed_at":"2026-06-16T09:16:06.925Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 166.985942] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.927Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7c322acaee9688559d4a489","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7c322acaee9688559d4a489: start","job_status":"start","queue_duration_s":3.157002,"scheduling_latency_s":3.157023} vm-test-run-nixbot-gitlab> gitlab # [ 167.061398] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7c322acaee9688559d4a489","created_at":"2026-06-16T09:16:03.687Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-a9272d875a46400e","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5d072a9d2b9baaeb2013b49cdc38c9a1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:03.770Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7c322acaee9688559d4a489: done: 0.015362 sec","job_status":"done","queue_duration_s":3.157002,"scheduling_latency_s":3.157023,"redis_calls":13,"redis_duration_s":0.002868,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015362,"completed_at":"2026-06-16T09:16:06.942Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 167.067044] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.943Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a7a9b708248ad916bfe13ece","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a7a9b708248ad916bfe13ece: start","job_status":"start","queue_duration_s":1.789174,"scheduling_latency_s":1.789196} vm-test-run-nixbot-gitlab> gitlab # [ 167.152452] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a7a9b708248ad916bfe13ece","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a7a9b708248ad916bfe13ece: done: 0.029209 sec","job_status":"done","queue_duration_s":1.789174,"scheduling_latency_s":1.789196,"redis_calls":13,"redis_duration_s":0.004322,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000803,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.006,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000509841,"duration_s":0.029209,"completed_at":"2026-06-16T09:16:06.973Z","load_balancing_strategy":"primary","db_duration_s":0.006166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 167.154238] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cfc7ee16f29fc889a0ac03eb","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cfc7ee16f29fc889a0ac03eb: start","job_status":"start","queue_duration_s":1.82022,"scheduling_latency_s":1.820241} vm-test-run-nixbot-gitlab> gitlab # [ 167.223421] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cfc7ee16f29fc889a0ac03eb","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cfc7ee16f29fc889a0ac03eb: done: 0.019303 sec","job_status":"done","queue_duration_s":1.82022,"scheduling_latency_s":1.820241,"redis_calls":10,"redis_duration_s":0.0043479999999999994,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002803,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019303,"completed_at":"2026-06-16T09:16:06.994Z","load_balancing_strategy":"primary","db_duration_s":0.002322,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 167.228444] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:06.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1a22517f43841516e005e700","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1a22517f43841516e005e700: start","job_status":"start","queue_duration_s":1.8419,"scheduling_latency_s":1.841923} vm-test-run-nixbot-gitlab> gitlab # [ 167.296257] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.016Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1a22517f43841516e005e700","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1a22517f43841516e005e700: done: 0.019411 sec","job_status":"done","queue_duration_s":1.8419,"scheduling_latency_s":1.841923,"redis_calls":10,"redis_duration_s":0.0044670000000000005,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001754,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002713,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019411,"completed_at":"2026-06-16T09:16:07.015Z","load_balancing_strategy":"primary","db_duration_s":0.002639,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 167.308115] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cab9753511a3579d09bf693f","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cab9753511a3579d09bf693f: start","job_status":"start","queue_duration_s":1.863223,"scheduling_latency_s":1.863245} vm-test-run-nixbot-gitlab> gitlab # [ 167.380370] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cab9753511a3579d09bf693f","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cab9753511a3579d09bf693f: done: 0.019886 sec","job_status":"done","queue_duration_s":1.863223,"scheduling_latency_s":1.863245,"redis_calls":10,"redis_duration_s":0.004482,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001806,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002676,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019886,"completed_at":"2026-06-16T09:16:07.037Z","load_balancing_strategy":"primary","db_duration_s":0.002709,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 167.387436] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7a5cf0c8c02ddaf81a58bf30","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7a5cf0c8c02ddaf81a58bf30: start","job_status":"start","queue_duration_s":1.884929,"scheduling_latency_s":1.884952} vm-test-run-nixbot-gitlab> gitlab # [ 167.457703] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.058Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7a5cf0c8c02ddaf81a58bf30","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7a5cf0c8c02ddaf81a58bf30: done: 0.01917 sec","job_status":"done","queue_duration_s":1.884929,"scheduling_latency_s":1.884952,"redis_calls":10,"redis_duration_s":0.0045850000000000005,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001814,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002771,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01917,"completed_at":"2026-06-16T09:16:07.058Z","load_balancing_strategy":"primary","db_duration_s":0.00271,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 167.468927] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6cfd82b1ed2bf30a75f3ccb1","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6cfd82b1ed2bf30a75f3ccb1: start","job_status":"start","queue_duration_s":1.905956,"scheduling_latency_s":1.905978} vm-test-run-nixbot-gitlab> gitlab # [ 167.488517] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6cfd82b1ed2bf30a75f3ccb1","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6cfd82b1ed2bf30a75f3ccb1: done: 0.019485 sec","job_status":"done","queue_duration_s":1.905956,"scheduling_latency_s":1.905978,"redis_calls":10,"redis_duration_s":0.004325,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001736,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002589,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019485,"completed_at":"2026-06-16T09:16:07.080Z","load_balancing_strategy":"primary","db_duration_s":0.002572,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 167.584223] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.082Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bd84a9125b557b59e848815a","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bd84a9125b557b59e848815a: start","job_status":"start","queue_duration_s":1.927874,"scheduling_latency_s":1.927897} vm-test-run-nixbot-gitlab> gitlab # [ 167.585804] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bd84a9125b557b59e848815a","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bd84a9125b557b59e848815a: done: 0.019206 sec","job_status":"done","queue_duration_s":1.927874,"scheduling_latency_s":1.927897,"redis_calls":10,"redis_duration_s":0.004319,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001687,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002632,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019206,"completed_at":"2026-06-16T09:16:07.101Z","load_balancing_strategy":"primary","db_duration_s":0.002669,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 167.629792] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.103Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0cf48a3406100b89b16427d2","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0cf48a3406100b89b16427d2: start","job_status":"start","queue_duration_s":1.949154,"scheduling_latency_s":1.949176} vm-test-run-nixbot-gitlab> gitlab # [ 167.649244] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0cf48a3406100b89b16427d2","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0cf48a3406100b89b16427d2: done: 0.019379 sec","job_status":"done","queue_duration_s":1.949154,"scheduling_latency_s":1.949176,"redis_calls":10,"redis_duration_s":0.004398,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001787,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002611,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019379,"completed_at":"2026-06-16T09:16:07.123Z","load_balancing_strategy":"primary","db_duration_s":0.002508,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 167.728404] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"42ec09bbc6f16bf4b27b9a4c","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-42ec09bbc6f16bf4b27b9a4c: start","job_status":"start","queue_duration_s":1.970462,"scheduling_latency_s":1.970484} vm-test-run-nixbot-gitlab> gitlab # [ 167.729727] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.145Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"42ec09bbc6f16bf4b27b9a4c","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-42ec09bbc6f16bf4b27b9a4c: done: 0.020171 sec","job_status":"done","queue_duration_s":1.970462,"scheduling_latency_s":1.970484,"redis_calls":10,"redis_duration_s":0.0046429999999999996,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001801,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002842,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020171,"completed_at":"2026-06-16T09:16:07.145Z","load_balancing_strategy":"primary","db_duration_s":0.002681,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 167.799368] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"443326fefaa31b8c8404aa20","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-443326fefaa31b8c8404aa20: start","job_status":"start","queue_duration_s":1.992391,"scheduling_latency_s":1.99246} vm-test-run-nixbot-gitlab> gitlab # [ 167.869468] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"443326fefaa31b8c8404aa20","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-443326fefaa31b8c8404aa20: done: 0.019235 sec","job_status":"done","queue_duration_s":1.992391,"scheduling_latency_s":1.99246,"redis_calls":10,"redis_duration_s":0.004139,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001646,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002493,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019235,"completed_at":"2026-06-16T09:16:07.166Z","load_balancing_strategy":"primary","db_duration_s":0.002601,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 167.871200] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3b9df7a0c558b905314051fb","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3b9df7a0c558b905314051fb: start","job_status":"start","queue_duration_s":2.01347,"scheduling_latency_s":2.013492} vm-test-run-nixbot-gitlab> gitlab # [ 167.886176] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3b9df7a0c558b905314051fb","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3b9df7a0c558b905314051fb: done: 0.019972 sec","job_status":"done","queue_duration_s":2.01347,"scheduling_latency_s":2.013492,"redis_calls":10,"redis_duration_s":0.004744,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001826,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002918,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019972,"completed_at":"2026-06-16T09:16:07.188Z","load_balancing_strategy":"primary","db_duration_s":0.002662,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 167.954958] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9510287fb89fd8c9a3accfb3","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9510287fb89fd8c9a3accfb3: start","job_status":"start","queue_duration_s":2.035318,"scheduling_latency_s":2.03534} vm-test-run-nixbot-gitlab> gitlab # [ 167.965312] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9510287fb89fd8c9a3accfb3","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9510287fb89fd8c9a3accfb3: done: 0.01997 sec","job_status":"done","queue_duration_s":2.035318,"scheduling_latency_s":2.03534,"redis_calls":10,"redis_duration_s":0.0044340000000000004,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00176,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002674,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01997,"completed_at":"2026-06-16T09:16:07.209Z","load_balancing_strategy":"primary","db_duration_s":0.002516,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 168.027263] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.211Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e75ae2244729142a78abf721","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e75ae2244729142a78abf721: start","job_status":"start","queue_duration_s":2.057192,"scheduling_latency_s":2.057213} vm-test-run-nixbot-gitlab> gitlab # [ 168.047285] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e75ae2244729142a78abf721","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e75ae2244729142a78abf721: done: 0.01984 sec","job_status":"done","queue_duration_s":2.057192,"scheduling_latency_s":2.057213,"redis_calls":10,"redis_duration_s":0.004575,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00173,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002845,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011185,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01984,"completed_at":"2026-06-16T09:16:07.231Z","load_balancing_strategy":"primary","db_duration_s":0.002354,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 168.158075] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1c7e93ab8ab369a4e8f8f02c","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1c7e93ab8ab369a4e8f8f02c: start","job_status":"start","queue_duration_s":2.078913,"scheduling_latency_s":2.078935} vm-test-run-nixbot-gitlab> gitlab # [ 168.158884] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1c7e93ab8ab369a4e8f8f02c","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1c7e93ab8ab369a4e8f8f02c: done: 0.020194 sec","job_status":"done","queue_duration_s":2.078913,"scheduling_latency_s":2.078935,"redis_calls":10,"redis_duration_s":0.004549,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001716,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002833,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020194,"completed_at":"2026-06-16T09:16:07.253Z","load_balancing_strategy":"primary","db_duration_s":0.003058,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 168.187818] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"064584ce0b031f560bb7d0e8","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-064584ce0b031f560bb7d0e8: start","job_status":"start","queue_duration_s":2.100942,"scheduling_latency_s":2.100964} vm-test-run-nixbot-gitlab> gitlab # [ 168.207495] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.275Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"064584ce0b031f560bb7d0e8","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-064584ce0b031f560bb7d0e8: done: 0.019562 sec","job_status":"done","queue_duration_s":2.100942,"scheduling_latency_s":2.100964,"redis_calls":10,"redis_duration_s":0.004266,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001639,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002627,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019562,"completed_at":"2026-06-16T09:16:07.275Z","load_balancing_strategy":"primary","db_duration_s":0.002711,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 168.302347] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.277Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"77638744f2921ddf96b0ebed","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-77638744f2921ddf96b0ebed: start","job_status":"start","queue_duration_s":2.122455,"scheduling_latency_s":2.122478} vm-test-run-nixbot-gitlab> gitlab # [ 168.303974] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"77638744f2921ddf96b0ebed","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-77638744f2921ddf96b0ebed: done: 0.024536 sec","job_status":"done","queue_duration_s":2.122455,"scheduling_latency_s":2.122478,"redis_calls":10,"redis_duration_s":0.008221,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.005345,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002876,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010527,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024536,"completed_at":"2026-06-16T09:16:07.301Z","load_balancing_strategy":"primary","db_duration_s":0.00398,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 168.382299] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"32a8e9d9d347442990b81580","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-32a8e9d9d347442990b81580: start","job_status":"start","queue_duration_s":2.148798,"scheduling_latency_s":2.148819} vm-test-run-nixbot-gitlab> gitlab # [ 168.383905] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"32a8e9d9d347442990b81580","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-32a8e9d9d347442990b81580: done: 0.021253 sec","job_status":"done","queue_duration_s":2.148798,"scheduling_latency_s":2.148819,"redis_calls":10,"redis_duration_s":0.003236,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001271,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001965,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021253,"completed_at":"2026-06-16T09:16:07.324Z","load_balancing_strategy":"primary","db_duration_s":0.003659,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 168.427343] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d4a38106bd3c039cd87e174f","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d4a38106bd3c039cd87e174f: start","job_status":"start","queue_duration_s":2.17209,"scheduling_latency_s":2.172135} vm-test-run-nixbot-gitlab> gitlab # [ 168.447191] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d4a38106bd3c039cd87e174f","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d4a38106bd3c039cd87e174f: done: 0.020608 sec","job_status":"done","queue_duration_s":2.17209,"scheduling_latency_s":2.172135,"redis_calls":10,"redis_duration_s":0.002383,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001446,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020608,"completed_at":"2026-06-16T09:16:07.347Z","load_balancing_strategy":"primary","db_duration_s":0.00183,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 168.509577] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4b6e64f7c31625a10bf9cd74","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4b6e64f7c31625a10bf9cd74: start","job_status":"start","queue_duration_s":2.194029,"scheduling_latency_s":2.194049} vm-test-run-nixbot-gitlab> gitlab # [ 168.590305] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4b6e64f7c31625a10bf9cd74","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4b6e64f7c31625a10bf9cd74: done: 0.016617 sec","job_status":"done","queue_duration_s":2.194029,"scheduling_latency_s":2.194049,"redis_calls":10,"redis_duration_s":0.002312,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001394,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016617,"completed_at":"2026-06-16T09:16:07.365Z","load_balancing_strategy":"primary","db_duration_s":0.001533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 168.592466] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"50f8f52e826ae0f4fef04b80","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-50f8f52e826ae0f4fef04b80: start","job_status":"start","queue_duration_s":2.212366,"scheduling_latency_s":2.212389} vm-test-run-nixbot-gitlab> gitlab # [ 168.608045] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"50f8f52e826ae0f4fef04b80","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-50f8f52e826ae0f4fef04b80: done: 0.016643 sec","job_status":"done","queue_duration_s":2.212366,"scheduling_latency_s":2.212389,"redis_calls":10,"redis_duration_s":0.002313,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001426,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016643,"completed_at":"2026-06-16T09:16:07.383Z","load_balancing_strategy":"primary","db_duration_s":0.001454,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 168.669473] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"441472553650fa7e9a651fc5","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-441472553650fa7e9a651fc5: start","job_status":"start","queue_duration_s":2.230343,"scheduling_latency_s":2.230365} vm-test-run-nixbot-gitlab> gitlab # [ 168.689152] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"441472553650fa7e9a651fc5","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-441472553650fa7e9a651fc5: done: 0.016645 sec","job_status":"done","queue_duration_s":2.230343,"scheduling_latency_s":2.230365,"redis_calls":10,"redis_duration_s":0.002378,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001485,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016645,"completed_at":"2026-06-16T09:16:07.401Z","load_balancing_strategy":"primary","db_duration_s":0.001585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 168.751187] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f31e8483c26e13164329ed6f","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f31e8483c26e13164329ed6f: start","job_status":"start","queue_duration_s":2.248301,"scheduling_latency_s":2.248324} vm-test-run-nixbot-gitlab> gitlab # [ 168.771288] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.419Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f31e8483c26e13164329ed6f","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f31e8483c26e13164329ed6f: done: 0.01651 sec","job_status":"done","queue_duration_s":2.248301,"scheduling_latency_s":2.248324,"redis_calls":10,"redis_duration_s":0.002254,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001331,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010833,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01651,"completed_at":"2026-06-16T09:16:07.419Z","load_balancing_strategy":"primary","db_duration_s":0.001602,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 168.832782] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.421Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6afe98e3326e17f9154f87ee","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6afe98e3326e17f9154f87ee: start","job_status":"start","queue_duration_s":2.266541,"scheduling_latency_s":2.266562} vm-test-run-nixbot-gitlab> gitlab # [ 168.852273] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.437Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6afe98e3326e17f9154f87ee","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6afe98e3326e17f9154f87ee: done: 0.016369 sec","job_status":"done","queue_duration_s":2.266541,"scheduling_latency_s":2.266562,"redis_calls":10,"redis_duration_s":0.002301,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001364,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016369,"completed_at":"2026-06-16T09:16:07.437Z","load_balancing_strategy":"primary","db_duration_s":0.001482,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 168.914224] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c4cf0e8f6893b055f5d8eb94","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c4cf0e8f6893b055f5d8eb94: start","job_status":"start","queue_duration_s":2.284225,"scheduling_latency_s":2.284246} vm-test-run-nixbot-gitlab> gitlab # [ 168.934329] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c4cf0e8f6893b055f5d8eb94","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c4cf0e8f6893b055f5d8eb94: done: 0.016435 sec","job_status":"done","queue_duration_s":2.284225,"scheduling_latency_s":2.284246,"redis_calls":10,"redis_duration_s":0.002325,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001408,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016435,"completed_at":"2026-06-16T09:16:07.455Z","load_balancing_strategy":"primary","db_duration_s":0.001574,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 169.028929] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5739dfa8541c2ee884380c20","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5739dfa8541c2ee884380c20: start","job_status":"start","queue_duration_s":2.301952,"scheduling_latency_s":2.301975} vm-test-run-nixbot-gitlab> gitlab # [ 169.030138] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.472Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5739dfa8541c2ee884380c20","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5739dfa8541c2ee884380c20: done: 0.016317 sec","job_status":"done","queue_duration_s":2.301952,"scheduling_latency_s":2.301975,"redis_calls":10,"redis_duration_s":0.002346,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001427,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016317,"completed_at":"2026-06-16T09:16:07.472Z","load_balancing_strategy":"primary","db_duration_s":0.001537,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 169.099100] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2f18a70d6d45a3620e2afc1b","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2f18a70d6d45a3620e2afc1b: start","job_status":"start","queue_duration_s":2.31966,"scheduling_latency_s":2.319681} vm-test-run-nixbot-gitlab> gitlab # [ 169.100456] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2f18a70d6d45a3620e2afc1b","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2f18a70d6d45a3620e2afc1b: done: 0.017265 sec","job_status":"done","queue_duration_s":2.31966,"scheduling_latency_s":2.319681,"redis_calls":10,"redis_duration_s":0.002268,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001393,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017265,"completed_at":"2026-06-16T09:16:07.491Z","load_balancing_strategy":"primary","db_duration_s":0.001756,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 169.168381] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bbcc98f0d4a13730b826e96a","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bbcc98f0d4a13730b826e96a: start","job_status":"start","queue_duration_s":2.338245,"scheduling_latency_s":2.338269} vm-test-run-nixbot-gitlab> gitlab # [ 169.170937] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.509Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bbcc98f0d4a13730b826e96a","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bbcc98f0d4a13730b826e96a: done: 0.016506 sec","job_status":"done","queue_duration_s":2.338245,"scheduling_latency_s":2.338269,"redis_calls":10,"redis_duration_s":0.0023380000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001466,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016506,"completed_at":"2026-06-16T09:16:07.509Z","load_balancing_strategy":"primary","db_duration_s":0.002054,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 169.232964] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0384326455eb25718c0f3d5a","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0384326455eb25718c0f3d5a: start","job_status":"start","queue_duration_s":2.356059,"scheduling_latency_s":2.356081} vm-test-run-nixbot-gitlab> gitlab # [ 169.314102] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.527Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0384326455eb25718c0f3d5a","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0384326455eb25718c0f3d5a: done: 0.017218 sec","job_status":"done","queue_duration_s":2.356059,"scheduling_latency_s":2.356081,"redis_calls":10,"redis_duration_s":0.002416,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001482,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017218,"completed_at":"2026-06-16T09:16:07.527Z","load_balancing_strategy":"primary","db_duration_s":0.001523,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 169.316088] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e6f762123a4132eac5370d14","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e6f762123a4132eac5370d14: start","job_status":"start","queue_duration_s":2.374718,"scheduling_latency_s":2.374741} vm-test-run-nixbot-gitlab> gitlab # [ 169.331295] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e6f762123a4132eac5370d14","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e6f762123a4132eac5370d14: done: 0.016578 sec","job_status":"done","queue_duration_s":2.374718,"scheduling_latency_s":2.374741,"redis_calls":10,"redis_duration_s":0.002284,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001373,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010492,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016578,"completed_at":"2026-06-16T09:16:07.545Z","load_balancing_strategy":"primary","db_duration_s":0.001577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 169.400186] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.547Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d14450243a91413f8d375bc3","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d14450243a91413f8d375bc3: start","job_status":"start","queue_duration_s":2.392657,"scheduling_latency_s":2.392678} vm-test-run-nixbot-gitlab> gitlab # [ 169.410329] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d14450243a91413f8d375bc3","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d14450243a91413f8d375bc3: done: 0.016552 sec","job_status":"done","queue_duration_s":2.392657,"scheduling_latency_s":2.392678,"redis_calls":10,"redis_duration_s":0.002295,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001331,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016552,"completed_at":"2026-06-16T09:16:07.563Z","load_balancing_strategy":"primary","db_duration_s":0.001585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 169.472307] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d859f7a7af6df45c650eb782","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d859f7a7af6df45c650eb782: start","job_status":"start","queue_duration_s":2.410568,"scheduling_latency_s":2.41059} vm-test-run-nixbot-gitlab> gitlab # [ 169.491949] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d859f7a7af6df45c650eb782","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d859f7a7af6df45c650eb782: done: 0.016427 sec","job_status":"done","queue_duration_s":2.410568,"scheduling_latency_s":2.41059,"redis_calls":10,"redis_duration_s":0.002306,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00138,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016427,"completed_at":"2026-06-16T09:16:07.581Z","load_balancing_strategy":"primary","db_duration_s":0.001512,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 169.602689] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4931f171cc4720e875b80ade","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4931f171cc4720e875b80ade: start","job_status":"start","queue_duration_s":2.428349,"scheduling_latency_s":2.428371} vm-test-run-nixbot-gitlab> gitlab # [ 169.604295] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4931f171cc4720e875b80ade","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4931f171cc4720e875b80ade: done: 0.016998 sec","job_status":"done","queue_duration_s":2.428349,"scheduling_latency_s":2.428371,"redis_calls":10,"redis_duration_s":0.002294,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001388,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016998,"completed_at":"2026-06-16T09:16:07.600Z","load_balancing_strategy":"primary","db_duration_s":0.001512,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 169.632439] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6e373ddbb4d1e5e3d1d71d63","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6e373ddbb4d1e5e3d1d71d63: start","job_status":"start","queue_duration_s":2.446725,"scheduling_latency_s":2.446746} vm-test-run-nixbot-gitlab> gitlab # [ 169.652927] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6e373ddbb4d1e5e3d1d71d63","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6e373ddbb4d1e5e3d1d71d63: done: 0.016352 sec","job_status":"done","queue_duration_s":2.446725,"scheduling_latency_s":2.446746,"redis_calls":10,"redis_duration_s":0.002264,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001344,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016352,"completed_at":"2026-06-16T09:16:07.617Z","load_balancing_strategy":"primary","db_duration_s":0.001629,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 169.714445] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4114183d589c7ba76304f082","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4114183d589c7ba76304f082: start","job_status":"start","queue_duration_s":2.464481,"scheduling_latency_s":2.464504} vm-test-run-nixbot-gitlab> gitlab # [ 169.791189] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4114183d589c7ba76304f082","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4114183d589c7ba76304f082: done: 0.016562 sec","job_status":"done","queue_duration_s":2.464481,"scheduling_latency_s":2.464504,"redis_calls":10,"redis_duration_s":0.002355,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000924,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001431,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016562,"completed_at":"2026-06-16T09:16:07.635Z","load_balancing_strategy":"primary","db_duration_s":0.001597,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 169.794141] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"47240d7c24bcbac519a7879d","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-47240d7c24bcbac519a7879d: start","job_status":"start","queue_duration_s":2.482305,"scheduling_latency_s":2.482328} vm-test-run-nixbot-gitlab> gitlab # [ 169.862148] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"47240d7c24bcbac519a7879d","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-47240d7c24bcbac519a7879d: done: 0.017055 sec","job_status":"done","queue_duration_s":2.482305,"scheduling_latency_s":2.482328,"redis_calls":10,"redis_duration_s":0.00233,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001349,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017055,"completed_at":"2026-06-16T09:16:07.654Z","load_balancing_strategy":"primary","db_duration_s":0.00161,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 169.873090] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.655Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"693e3af2432fa0333d65daa3","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-693e3af2432fa0333d65daa3: start","job_status":"start","queue_duration_s":2.500712,"scheduling_latency_s":2.500734} vm-test-run-nixbot-gitlab> gitlab # [ 169.892844] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"693e3af2432fa0333d65daa3","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-693e3af2432fa0333d65daa3: done: 0.016299 sec","job_status":"done","queue_duration_s":2.500712,"scheduling_latency_s":2.500734,"redis_calls":10,"redis_duration_s":0.002261,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001341,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016299,"completed_at":"2026-06-16T09:16:07.671Z","load_balancing_strategy":"primary","db_duration_s":0.00153,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 169.954047] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bdae9adc97322f5f03e44823","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bdae9adc97322f5f03e44823: start","job_status":"start","queue_duration_s":2.518381,"scheduling_latency_s":2.518402} vm-test-run-nixbot-gitlab> gitlab # [ 169.975647] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bdae9adc97322f5f03e44823","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bdae9adc97322f5f03e44823: done: 0.016621 sec","job_status":"done","queue_duration_s":2.518381,"scheduling_latency_s":2.518402,"redis_calls":10,"redis_duration_s":0.002405,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00103,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001375,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01089,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016621,"completed_at":"2026-06-16T09:16:07.689Z","load_balancing_strategy":"primary","db_duration_s":0.001562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 170.038850] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4456846293c2ccd3af650807","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4456846293c2ccd3af650807: start","job_status":"start","queue_duration_s":2.536725,"scheduling_latency_s":2.536748} vm-test-run-nixbot-gitlab> gitlab # [ 170.058252] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4456846293c2ccd3af650807","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4456846293c2ccd3af650807: done: 0.016383 sec","job_status":"done","queue_duration_s":2.536725,"scheduling_latency_s":2.536748,"redis_calls":10,"redis_duration_s":0.002257,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001334,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016383,"completed_at":"2026-06-16T09:16:07.707Z","load_balancing_strategy":"primary","db_duration_s":0.0017,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 170.184964] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.709Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0bcc1f8ee1dd64e1a0a45beb","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0bcc1f8ee1dd64e1a0a45beb: start","job_status":"start","queue_duration_s":2.554498,"scheduling_latency_s":2.554521} vm-test-run-nixbot-gitlab> gitlab # [ 170.186471] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0bcc1f8ee1dd64e1a0a45beb","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0bcc1f8ee1dd64e1a0a45beb: done: 0.016492 sec","job_status":"done","queue_duration_s":2.554498,"scheduling_latency_s":2.554521,"redis_calls":10,"redis_duration_s":0.002357,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001461,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016492,"completed_at":"2026-06-16T09:16:07.725Z","load_balancing_strategy":"primary","db_duration_s":0.00158,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 170.254903] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c634cdc8d0d2336e650f04cd","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c634cdc8d0d2336e650f04cd: start","job_status":"start","queue_duration_s":2.572316,"scheduling_latency_s":2.572338} vm-test-run-nixbot-gitlab> gitlab # [ 170.256281] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c634cdc8d0d2336e650f04cd","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c634cdc8d0d2336e650f04cd: done: 0.01691 sec","job_status":"done","queue_duration_s":2.572316,"scheduling_latency_s":2.572338,"redis_calls":10,"redis_duration_s":0.002333,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001423,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01691,"completed_at":"2026-06-16T09:16:07.743Z","load_balancing_strategy":"primary","db_duration_s":0.001575,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 170.326136] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b037a79bef478ae6a61738a2","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b037a79bef478ae6a61738a2: start","job_status":"start","queue_duration_s":2.590604,"scheduling_latency_s":2.590628} vm-test-run-nixbot-gitlab> gitlab # [ 170.327816] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.761Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b037a79bef478ae6a61738a2","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b037a79bef478ae6a61738a2: done: 0.016677 sec","job_status":"done","queue_duration_s":2.590604,"scheduling_latency_s":2.590628,"redis_calls":10,"redis_duration_s":0.0023049999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00138,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016677,"completed_at":"2026-06-16T09:16:07.761Z","load_balancing_strategy":"primary","db_duration_s":0.001631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 170.415052] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a7309714b5f202f59fb7c560","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a7309714b5f202f59fb7c560: start","job_status":"start","queue_duration_s":2.609133,"scheduling_latency_s":2.609157} vm-test-run-nixbot-gitlab> gitlab # [ 170.416429] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.782Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a7309714b5f202f59fb7c560","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a7309714b5f202f59fb7c560: done: 0.018988 sec","job_status":"done","queue_duration_s":2.609133,"scheduling_latency_s":2.609157,"redis_calls":10,"redis_duration_s":0.002918,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001421,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018988,"completed_at":"2026-06-16T09:16:07.782Z","load_balancing_strategy":"primary","db_duration_s":0.003618,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 170.486819] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"46f43173be4e0772e501e52e","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-46f43173be4e0772e501e52e: start","job_status":"start","queue_duration_s":2.629518,"scheduling_latency_s":2.629542} vm-test-run-nixbot-gitlab> gitlab # [ 170.488389] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"46f43173be4e0772e501e52e","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-46f43173be4e0772e501e52e: done: 0.018187 sec","job_status":"done","queue_duration_s":2.629518,"scheduling_latency_s":2.629542,"redis_calls":10,"redis_duration_s":0.0024000000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001427,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018187,"completed_at":"2026-06-16T09:16:07.802Z","load_balancing_strategy":"primary","db_duration_s":0.001482,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 170.578294] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.803Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"66c46200ef63e3a9dc4bf892","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-66c46200ef63e3a9dc4bf892: start","job_status":"start","queue_duration_s":2.649027,"scheduling_latency_s":2.649049} vm-test-run-nixbot-gitlab> gitlab # [ 170.580157] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"66c46200ef63e3a9dc4bf892","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-66c46200ef63e3a9dc4bf892: done: 0.016069 sec","job_status":"done","queue_duration_s":2.649027,"scheduling_latency_s":2.649049,"redis_calls":10,"redis_duration_s":0.002289,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001401,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010582,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016069,"completed_at":"2026-06-16T09:16:07.819Z","load_balancing_strategy":"primary","db_duration_s":0.001498,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 170.664392] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4e15549a828201a4579cef89","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4e15549a828201a4579cef89: start","job_status":"start","queue_duration_s":2.666566,"scheduling_latency_s":2.666588} vm-test-run-nixbot-gitlab> gitlab # [ 170.668746] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4e15549a828201a4579cef89","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4e15549a828201a4579cef89: done: 0.169188 sec","job_status":"done","queue_duration_s":2.666566,"scheduling_latency_s":2.666588,"redis_calls":10,"redis_duration_s":0.0028,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001434,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001366,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.169188,"completed_at":"2026-06-16T09:16:07.990Z","load_balancing_strategy":"primary","db_duration_s":0.001509,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 170.678552] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:07.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e85b2791b3cbad4858a531da","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e85b2791b3cbad4858a531da: start","job_status":"start","queue_duration_s":2.83923,"scheduling_latency_s":2.839294} vm-test-run-nixbot-gitlab> gitlab # [ 170.699255] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:08.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e85b2791b3cbad4858a531da","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e85b2791b3cbad4858a531da: done: 0.039786 sec","job_status":"done","queue_duration_s":2.83923,"scheduling_latency_s":2.839294,"redis_calls":10,"redis_duration_s":0.0035310000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001429,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002102,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039786,"completed_at":"2026-06-16T09:16:08.033Z","load_balancing_strategy":"primary","db_duration_s":0.00256,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 170.775352] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:08.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"528c262ded230d525df831b6","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-528c262ded230d525df831b6: start","job_status":"start","queue_duration_s":2.881613,"scheduling_latency_s":2.881657} vm-test-run-nixbot-gitlab> gitlab # [ 170.782241] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:08.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"528c262ded230d525df831b6","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-528c262ded230d525df831b6: done: 0.018408 sec","job_status":"done","queue_duration_s":2.881613,"scheduling_latency_s":2.881657,"redis_calls":10,"redis_duration_s":0.002357,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000924,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001433,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018408,"completed_at":"2026-06-16T09:16:08.054Z","load_balancing_strategy":"primary","db_duration_s":0.001734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 170.869267] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:08.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8dd467a05e77674c6d18f580","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8dd467a05e77674c6d18f580: start","job_status":"start","queue_duration_s":2.901309,"scheduling_latency_s":2.901332} vm-test-run-nixbot-gitlab> gitlab # [ 170.870328] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:08.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8dd467a05e77674c6d18f580","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8dd467a05e77674c6d18f580: done: 0.016469 sec","job_status":"done","queue_duration_s":2.901309,"scheduling_latency_s":2.901332,"redis_calls":10,"redis_duration_s":0.002343,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001456,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016469,"completed_at":"2026-06-16T09:16:08.072Z","load_balancing_strategy":"primary","db_duration_s":0.001588,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 170.951379] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:08.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"14a09ba3272c767a4547f5df","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-14a09ba3272c767a4547f5df: start","job_status":"start","queue_duration_s":2.919093,"scheduling_latency_s":2.919116} vm-test-run-nixbot-gitlab> gitlab # [ 170.952987] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:08.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"14a09ba3272c767a4547f5df","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-14a09ba3272c767a4547f5df: done: 0.016194 sec","job_status":"done","queue_duration_s":2.919093,"scheduling_latency_s":2.919116,"redis_calls":10,"redis_duration_s":0.002211,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001338,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016194,"completed_at":"2026-06-16T09:16:08.089Z","load_balancing_strategy":"primary","db_duration_s":0.001477,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 171.049890] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:08.091Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2add0b8b9aed2614924f033a","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2add0b8b9aed2614924f033a: start","job_status":"start","queue_duration_s":2.936877,"scheduling_latency_s":2.936899} vm-test-run-nixbot-gitlab> gitlab # [ 171.051394] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:08.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2add0b8b9aed2614924f033a","created_at":"2026-06-16T09:16:05.067Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-ce6ead29f4d24e46","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"85f1babf999eade50a5bc37af13c278d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:05.154Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2add0b8b9aed2614924f033a: done: 0.016213 sec","job_status":"done","queue_duration_s":2.936877,"scheduling_latency_s":2.936899,"redis_calls":10,"redis_duration_s":0.0022660000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001345,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016213,"completed_at":"2026-06-16T09:16:08.107Z","load_balancing_strategy":"primary","db_duration_s":0.001488,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 171.414110] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 8852998045 wd_nsec: 8852993814 vm-test-run-nixbot-gitlab> gitlab # [ 183.999651] setup-gitlab-start[2793]: W, [2026-06-16T09:16:27.881590 #2793] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. vm-test-run-nixbot-gitlab> gitlab # [ 209.951629] runuser[2792]: pam_unix(runuser:session): session closed for user gitlab vm-test-run-nixbot-gitlab> gitlab # [ 209.959538] setup-gitlab-start[3039]: ++ jq -r .token vm-test-run-nixbot-gitlab> gitlab # [ 209.962490] setup-gitlab-start[3040]: +++ date -d '+7 days' +%F vm-test-run-nixbot-gitlab> gitlab # [ 209.965499] setup-gitlab-start[3038]: ++ curl -fs -X POST -H 'Authorization: Bearer 0d59f68ec4dcbedcefb3add85736a9a13d57010605fcd88c0954a364817ec887' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-06-23"}' http://gitlab/api/v4/users/1/personal_access_tokens vm-test-run-nixbot-gitlab> gitlab # [ 210.619577] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:54.500Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"d941db8fbe9258fdaf7e9fb5","created_at":"2026-06-16T09:16:54.470Z","trace_propagation_headers":{"sentry-trace":"a1206a821b7b46cab787b3bbcb83607a-8475d8124ba34848","baggage":"sentry-trace_id=a1206a821b7b46cab787b3bbcb83607a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGAQBSKZRVYY46762XQM9","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:35ba3b039afb466fbba0192235e5d48a7add31112fab869c33bfc281eba7de8f","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:54.495Z","job_size_bytes":451,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-d941db8fbe9258fdaf7e9fb5: start","job_status":"start","queue_duration_s":0.004491,"scheduling_latency_s":0.004574} vm-test-run-nixbot-gitlab> gitlab # [ 210.688319] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:16:54 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 701 vm-test-run-nixbot-gitlab> gitlab # [ 210.696937] setup-gitlab-start[2749]: + TOKEN=glpat-qurTyD742jXG5bfscRpKU286MQp1OjEH.01.0w0dp5cu7 vm-test-run-nixbot-gitlab> gitlab # [ 210.698636] setup-gitlab-start[2749]: + test -n glpat-qurTyD742jXG5bfscRpKU286MQp1OjEH.01.0w0dp5cu7 vm-test-run-nixbot-gitlab> gitlab # [ 210.700213] setup-gitlab-start[2749]: + test glpat-qurTyD742jXG5bfscRpKU286MQp1OjEH.01.0w0dp5cu7 '!=' null vm-test-run-nixbot-gitlab> gitlab # [ 210.701852] setup-gitlab-start[2749]: + mkdir -p /var/lib/secrets vm-test-run-nixbot-gitlab> gitlab # [ 210.708068] setup-gitlab-start[2749]: + echo glpat-qurTyD742jXG5bfscRpKU286MQp1OjEH.01.0w0dp5cu7 vm-test-run-nixbot-gitlab> gitlab # [ 210.709174] setup-gitlab-start[2749]: + chmod 644 /var/lib/secrets/gitlab-token vm-test-run-nixbot-gitlab> gitlab # [ 210.713230] setup-gitlab-start[2749]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-qurTyD742jXG5bfscRpKU286MQp1OjEH.01.0w0dp5cu7' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects vm-test-run-nixbot-gitlab> gitlab # [ 210.793240] sidekiq-cluster[2167]: {"severity":"WARN","time":"2026-06-16T09:16:54.675Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"d941db8fbe9258fdaf7e9fb5","created_at":"2026-06-16T09:16:54.470Z","trace_propagation_headers":{"sentry-trace":"a1206a821b7b46cab787b3bbcb83607a-8475d8124ba34848","baggage":"sentry-trace_id=a1206a821b7b46cab787b3bbcb83607a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGAQBSKZRVYY46762XQM9","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:35ba3b039afb466fbba0192235e5d48a7add31112fab869c33bfc281eba7de8f","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:54.495Z","job_size_bytes":451,"pid":2167,"sidekiq_tid":"lymf","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-d941db8fbe9258fdaf7e9fb5: fail: 0.174889 sec","job_status":"fail","queue_duration_s":0.004491,"scheduling_latency_s":0.004574,"redis_calls":19,"redis_duration_s":0.008447,"redis_read_bytes":653,"redis_write_bytes":2107,"redis_feature_flag_calls":11,"redis_feature_flag_duration_s":0.00463,"redis_feature_flag_read_bytes":646,"redis_feature_flag_write_bytes":1223,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002803,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.112252,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000849828,"duration_s":0.174889,"completed_at":"2026-06-16T09:16:54.674Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.036476,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 212.025562] gitaly[1373]: time="2026-06-16T09:16:55.907Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-16T09:17:05.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:55.906" pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.028747] gitaly[1373]: time="2026-06-16T09:16:55.907Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-16T09:17:05.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:55.906" grpc.time_ms=0.77 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.222898] gitaly[1373]: time="2026-06-16T09:16:56.105Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-16T09:17:06.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.105" pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.226503] gitaly[1373]: time="2026-06-16T09:16:56.106Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-16T09:17:06.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.105" grpc.time_ms=0.497 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.280143] gitaly[1373]: time="2026-06-16T09:16:56.162Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 diskcache=8033d372-7147-4225-87e8-6a51fd9f964c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-16T09:17:06.019" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.120" pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.283268] gitaly[1373]: time="2026-06-16T09:16:56.163Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=324 command.oublock=32 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-16T09:17:06.019" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.120" grpc.time_ms=42.488 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.298586] gitaly[1373]: time="2026-06-16T09:16:56.181Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-16T09:17:06.080" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.180" pid=1373 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.300747] gitaly[1373]: time="2026-06-16T09:16:56.181Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-16T09:17:06.080" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.180" grpc.time_ms=0.827 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.371409] gitaly[1373]: time="2026-06-16T09:16:56.254Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=62 command.spawn_token_fork_ms=54 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-16T09:17:06.088" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.189" grpc.time_ms=64.254 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.394882] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:56.276Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"cc6453ff35e8802109f4fb76","created_at":"2026-06-16T09:16:56.265Z","trace_propagation_headers":{"sentry-trace":"b73702ebca3c47a3a18230d34258530a-f8a2868284e6459c","baggage":"sentry-trace_id=b73702ebca3c47a3a18230d34258530a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGBEM1B3868M9A8J91403","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:56.274Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-cc6453ff35e8802109f4fb76: start","job_status":"start","queue_duration_s":0.002066,"scheduling_latency_s":0.002121} vm-test-run-nixbot-gitlab> gitlab # [ 212.419366] gitaly[1373]: time="2026-06-16T09:16:56.294Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-16T09:17:06.193" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.293" pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.421190] gitaly[1373]: time="2026-06-16T09:16:56.294Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-16T09:17:06.193" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.293" grpc.time_ms=0.366 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.442494] gitaly[1373]: time="2026-06-16T09:16:56.325Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 diskcache=a4e3b7fd-57c6-495c-b8d9-1fcb9caa40f7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-16T09:17:06.195" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.296" pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.444100] gitaly[1373]: time="2026-06-16T09:16:56.325Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=326 command.oublock=32 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-06-16T09:17:06.195" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.296" grpc.time_ms=29.514 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.496297] gitaly[1373]: time="2026-06-16T09:16:56.379Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-16T09:17:06.277" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.378" pid=1373 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.497554] gitaly[1373]: time="2026-06-16T09:16:56.379Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-16T09:17:06.277" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.378" grpc.time_ms=0.918 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.544793] gitaly[1373]: time="2026-06-16T09:16:56.427Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-16T09:17:06.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.419" grpc.time_ms=7.745 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.551165] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:56.433Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"cc6453ff35e8802109f4fb76","created_at":"2026-06-16T09:16:56.265Z","trace_propagation_headers":{"sentry-trace":"b73702ebca3c47a3a18230d34258530a-f8a2868284e6459c","baggage":"sentry-trace_id=b73702ebca3c47a3a18230d34258530a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGBEM1B3868M9A8J91403","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:56.274Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-cc6453ff35e8802109f4fb76: done: 0.157717 sec","job_status":"done","queue_duration_s":0.002066,"scheduling_latency_s":0.002121,"redis_calls":16,"redis_duration_s":0.00842,"redis_read_bytes":268,"redis_write_bytes":1956,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002674,"redis_feature_flag_read_bytes":260,"redis_feature_flag_write_bytes":791,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003497,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002249,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050798,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.039701694,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.157717,"completed_at":"2026-06-16T09:16:56.433Z","load_balancing_strategy":"primary","db_duration_s":0.015206,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 212.632464] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:56.435Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"18cc3c735e9eaf9c7ee447c5","created_at":"2026-06-16T09:16:56.283Z","trace_propagation_headers":{"sentry-trace":"b73702ebca3c47a3a18230d34258530a-f8a2868284e6459c","baggage":"sentry-trace_id=b73702ebca3c47a3a18230d34258530a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGBEM1B3868M9A8J91403","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:56.289Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-18cc3c735e9eaf9c7ee447c5: start","job_status":"start","queue_duration_s":0.146286,"scheduling_latency_s":0.146316} vm-test-run-nixbot-gitlab> gitlab # [ 212.633896] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:56.515Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"18cc3c735e9eaf9c7ee447c5","created_at":"2026-06-16T09:16:56.283Z","trace_propagation_headers":{"sentry-trace":"b73702ebca3c47a3a18230d34258530a-f8a2868284e6459c","baggage":"sentry-trace_id=b73702ebca3c47a3a18230d34258530a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGBEM1B3868M9A8J91403","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:56.289Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-18cc3c735e9eaf9c7ee447c5: done: 0.079807 sec","job_status":"done","queue_duration_s":0.146286,"scheduling_latency_s":0.146316,"redis_calls":14,"redis_duration_s":0.003661,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000724,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001617,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00132,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.058876197,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.079807,"completed_at":"2026-06-16T09:16:56.515Z","load_balancing_strategy":"primary","db_duration_s":0.019046,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 212.730951] gitaly[1373]: time="2026-06-16T09:16:56.613Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-16T09:17:06.202" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.613" grpc.time_ms=0.235 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 212.744926] gitaly[1373]: time="2026-06-16T09:16:56.627Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-16T09:17:06.216" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.627" grpc.time_ms=0.323 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 213.011496] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:56.893Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"5b2186876f3f21f0122747bf","created_at":"2026-06-16T09:16:56.882Z","trace_propagation_headers":{"sentry-trace":"b73702ebca3c47a3a18230d34258530a-f8a2868284e6459c","baggage":"sentry-trace_id=b73702ebca3c47a3a18230d34258530a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGBEM1B3868M9A8J91403","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:56.889Z","job_size_bytes":5,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-5b2186876f3f21f0122747bf: start","job_status":"start","queue_duration_s":0.003466,"scheduling_latency_s":0.00355} vm-test-run-nixbot-gitlab> gitlab # [ 213.083400] gitaly[1373]: time="2026-06-16T09:16:56.965Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGBEM1B3868M9A8J91403 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-16T09:17:06.055" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:56.956" grpc.time_ms=8.008 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 213.202217] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:57.084Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"5b2186876f3f21f0122747bf","created_at":"2026-06-16T09:16:56.882Z","trace_propagation_headers":{"sentry-trace":"b73702ebca3c47a3a18230d34258530a-f8a2868284e6459c","baggage":"sentry-trace_id=b73702ebca3c47a3a18230d34258530a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGBEM1B3868M9A8J91403","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:56.889Z","job_size_bytes":5,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-5b2186876f3f21f0122747bf: done: 0.191452 sec","job_status":"done","queue_duration_s":0.003466,"scheduling_latency_s":0.00355,"redis_calls":12,"redis_duration_s":0.005741,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000791,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003776,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.090367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000614044,"duration_s":0.191452,"completed_at":"2026-06-16T09:16:57.084Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024107,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 213.306629] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:57.189Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"90e0b9fe08502b66048aaf82","created_at":"2026-06-16T09:16:56.917Z","trace_propagation_headers":{"sentry-trace":"b73702ebca3c47a3a18230d34258530a-f8a2868284e6459c","baggage":"sentry-trace_id=b73702ebca3c47a3a18230d34258530a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGBEM1B3868M9A8J91403","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:56.922Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-90e0b9fe08502b66048aaf82: start","job_status":"start","queue_duration_s":0.26631,"scheduling_latency_s":0.266362} vm-test-run-nixbot-gitlab> gitlab # [ 213.549108] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:16:57 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 2818 vm-test-run-nixbot-gitlab> gitlab # [ 213.559600] setup-gitlab-start[2749]: + mkdir -p /tmp/test-flake vm-test-run-nixbot-gitlab> gitlab # [ 213.565932] setup-gitlab-start[2749]: + cd /tmp/test-flake vm-test-run-nixbot-gitlab> gitlab # [ 213.567868] setup-gitlab-start[2749]: + git init -b master vm-test-run-nixbot-gitlab> gitlab # [ 213.585908] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:57.468Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"90e0b9fe08502b66048aaf82","created_at":"2026-06-16T09:16:56.917Z","trace_propagation_headers":{"sentry-trace":"b73702ebca3c47a3a18230d34258530a-f8a2868284e6459c","baggage":"sentry-trace_id=b73702ebca3c47a3a18230d34258530a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGBEM1B3868M9A8J91403","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:56.922Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-90e0b9fe08502b66048aaf82: done: 0.279207 sec","job_status":"done","queue_duration_s":0.26631,"scheduling_latency_s":0.266362,"redis_calls":11,"redis_duration_s":0.0058790000000000005,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.002084,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002952,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000843,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.184,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.184,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.146247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000483302,"duration_s":0.279207,"completed_at":"2026-06-16T09:16:57.468Z","load_balancing_strategy":"primary","db_duration_s":0.096768,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 213.590285] setup-gitlab-start[3091]: Initialized empty Git repository in /tmp/test-flake/.git/ vm-test-run-nixbot-gitlab> gitlab # [ 213.593389] setup-gitlab-start[2749]: + git config user.name test vm-test-run-nixbot-gitlab> gitlab # [ 213.595634] setup-gitlab-start[2749]: + git config user.email test@example.com vm-test-run-nixbot-gitlab> gitlab # [ 213.597775] setup-gitlab-start[2749]: + cat vm-test-run-nixbot-gitlab> gitlab # [ 213.600765] setup-gitlab-start[2749]: + git add flake.nix vm-test-run-nixbot-gitlab> gitlab # [ 213.603477] setup-gitlab-start[2749]: + git commit -m 'initial commit' vm-test-run-nixbot-gitlab> gitlab # [ 213.613112] setup-gitlab-start[3096]: [master (root-commit) ed9f2d2] initial commit vm-test-run-nixbot-gitlab> gitlab # [ 213.614433] setup-gitlab-start[3096]: 1 file changed, 10 insertions(+) vm-test-run-nixbot-gitlab> gitlab # [ 213.615253] setup-gitlab-start[3096]: create mode 100644 flake.nix vm-test-run-nixbot-gitlab> gitlab # [ 213.616210] setup-gitlab-start[2749]: + git remote add origin http://oauth2:glpat-qurTyD742jXG5bfscRpKU286MQp1OjEH.01.0w0dp5cu7@gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 213.617576] setup-gitlab-start[2749]: + git push -u origin master vm-test-run-nixbot-gitlab> gitlab # [ 213.688883] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:57.571Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"8d08e0c2cc0865ef87e4b814","created_at":"2026-06-16T09:16:56.925Z","trace_propagation_headers":{"sentry-trace":"b73702ebca3c47a3a18230d34258530a-f8a2868284e6459c","baggage":"sentry-trace_id=b73702ebca3c47a3a18230d34258530a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGBEM1B3868M9A8J91403","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:56.930Z","job_size_bytes":89,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-8d08e0c2cc0865ef87e4b814: start","job_status":"start","queue_duration_s":0.640883,"scheduling_latency_s":0.640942} vm-test-run-nixbot-gitlab> gitlab # [ 213.753506] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:57.636Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"8d08e0c2cc0865ef87e4b814","created_at":"2026-06-16T09:16:56.925Z","trace_propagation_headers":{"sentry-trace":"b73702ebca3c47a3a18230d34258530a-f8a2868284e6459c","baggage":"sentry-trace_id=b73702ebca3c47a3a18230d34258530a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGBEM1B3868M9A8J91403","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:56.930Z","job_size_bytes":89,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-8d08e0c2cc0865ef87e4b814: done: 0.064976 sec","job_status":"done","queue_duration_s":0.640883,"scheduling_latency_s":0.640942,"redis_calls":11,"redis_duration_s":0.002989,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000712,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001796,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004576,"duration_s":0.064976,"completed_at":"2026-06-16T09:16:57.636Z","load_balancing_strategy":"primary","db_duration_s":0.02131,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 213.812368] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:16:57 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 173 vm-test-run-nixbot-gitlab> gitlab # [ 213.999894] gitaly[1373]: time="2026-06-16T09:16:57.882Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEF232FGDWXSWZV6T06S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:16:57.864" grpc.time_ms=17.948 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.048570] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:16:57 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 232 vm-test-run-nixbot-gitlab> gitlab # [ 214.329389] gitaly[1373]: time="2026-06-16T09:16:58.211Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-16T09:17:08.097" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:58.200" grpc.time_ms=10.787 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.361183] gitaly[1373]: time="2026-06-16T09:16:58.244Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=330 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-16T09:17:28.034" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-16T09:16:58.234" grpc.time_ms=9.307 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.383935] gitaly[1373]: time="2026-06-16T09:16:58.266Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B duration_ms=131ns method=POST pid=1373 status=200 url="http://unix/api/v4/internal/allowed" vm-test-run-nixbot-gitlab> gitlab # [ 214.436426] gitaly[1373]: time="2026-06-16T09:16:58.319Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B duration_ms=51ns method=POST pid=1373 status=200 url="http://unix/api/v4/internal/pre_receive" vm-test-run-nixbot-gitlab> gitlab # [ 214.438183] gitaly[1373]: time="2026-06-16T09:16:58.319Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:16:58.134" grpc.time_ms=185.271 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.486728] gitaly[1373]: time="2026-06-16T09:16:58.369Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:16:58.368" grpc.time_ms=0.68 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.534569] gitaly[1373]: time="2026-06-16T09:16:58.417Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:16:58.416" grpc.time_ms=0.651 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.576603] gitaly[1373]: time="2026-06-16T09:16:58.459Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:16:58.458" grpc.time_ms=0.531 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.688884] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:58.570Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"48a3f2e0ae96c8d99143cba7","created_at":"2026-06-16T09:16:58.540Z","trace_propagation_headers":{"sentry-trace":"cd52cffba58d4d60b42ee524ec7eb63d-639fd0f7170b44c9","baggage":"sentry-trace_id=cd52cffba58d4d60b42ee524ec7eb63d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGEPX2ZW25XMXT68M1K8B","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:58.566Z","job_size_bytes":185,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-48a3f2e0ae96c8d99143cba7: start","job_status":"start","queue_duration_s":0.004118,"scheduling_latency_s":0.00417} vm-test-run-nixbot-gitlab> gitlab # [ 214.707606] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:58.590Z","message":"changes: \"0000000000000000000000000000000000000000 ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf refs/heads/master\\n\""} vm-test-run-nixbot-gitlab> gitlab # [ 214.736569] gitaly[1373]: time="2026-06-16T09:16:58.619Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-16T09:17:08.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:58.608" grpc.time_ms=10.535 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.759110] gitaly[1373]: time="2026-06-16T09:16:58.641Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-16T09:17:08.000" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:58.630" grpc.time_ms=10.8 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.764151] gitaly[1373]: time="2026-06-16T09:16:58.646Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-16T09:17:08.037" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:16:58.637" grpc.time_ms=8.853 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.809949] gitaly[1373]: time="2026-06-16T09:16:58.692Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1568 command.oublock=0 command.real_time_ms=35 command.spawn_token_fork_ms=22 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-16T09:17:08.052" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:16:58.653" grpc.time_ms=39.158 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.812162] gitaly[1373]: time="2026-06-16T09:16:58.693Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1571 command.oublock=0 command.real_time_ms=33 command.spawn_token_fork_ms=20 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-16T09:17:08.024" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:16:58.654" grpc.time_ms=38.692 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.830177] gitaly[1373]: time="2026-06-16T09:16:58.712Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B duration_ms=207ns method=POST pid=1373 status=200 url="http://unix/api/v4/internal/post_receive" vm-test-run-nixbot-gitlab> gitlab # [ 214.833112] gitaly[1373]: time="2026-06-16T09:16:58.715Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:16:58.504" grpc.time_ms=211.109 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.885560] gitaly[1373]: time="2026-06-16T09:16:58.768Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=36 command.spawn_token_fork_ms=31 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-16T09:17:08.127" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:16:58.728" grpc.time_ms=40.184 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.888312] gitaly[1373]: time="2026-06-16T09:16:58.771Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B diskcache=e681a311-c6f7-4b46-b595-2f01ecddc067 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:16:58.034" pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.892341] gitaly[1373]: time="2026-06-16T09:16:58.771Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=262 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13594 command.oublock=32 command.real_time_ms=724 command.spawn_token_fork_ms=1 command.system_time_ms=169 command.user_time_ms=93 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:16:58.034" grpc.time_ms=737.252 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.893430] setup-gitlab-start[3100]: To http://gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # [ 214.894812] setup-gitlab-start[3100]: * [new branch] master -> master vm-test-run-nixbot-gitlab> gitlab # [ 214.896634] gitlab-workhorse[1288]: time="2026-06-16T09:16:58Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 vm-test-run-nixbot-gitlab> gitlab # [ 214.897866] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:16:58 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 822 vm-test-run-nixbot-gitlab> gitlab # [ 214.899075] setup-gitlab-start[3100]: branch 'master' set up to track 'origin/master'. vm-test-run-nixbot-gitlab> gitlab # [ 214.932049] gitaly[1373]: time="2026-06-16T09:16:58.814Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3942 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:16:58.814" grpc.time_ms=0.326 pid=1373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 214.953929] gitaly[1373]: time="2026-06-16T09:16:58.836Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3944 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:16:58.836" grpc.time_ms=0.259 pid=1373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 214.966492] gitaly[1373]: time="2026-06-16T09:16:58.849Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B diskcache=9fb8ede7-f67d-4e15-afb0-505a41d74556 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-16T09:17:08.190" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:58.790" pid=1373 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.967455] gitaly[1373]: time="2026-06-16T09:16:58.849Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=53 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4656 command.oublock=8 command.real_time_ms=49 command.spawn_token_fork_ms=0 command.system_time_ms=36 command.user_time_ms=17 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-16T09:17:08.190" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:58.790" grpc.time_ms=58.755 pid=1373 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.986413] gitaly[1373]: time="2026-06-16T09:16:58.869Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-06-16T09:17:08.268" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:58.868" grpc.time_ms=0.183 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 214.999842] gitaly[1373]: time="2026-06-16T09:16:58.882Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=938 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-06-16T09:17:08.273" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:16:58.874" grpc.time_ms=8.496 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 215.033301] systemd[1]: Finished setup-gitlab.service. vm-test-run-nixbot-gitlab> gitlab # [ 215.040374] systemd[1]: Starting nixbot CI service... vm-test-run-nixbot-gitlab> gitlab # [ 215.073110] gitaly[1373]: time="2026-06-16T09:16:58.955Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-06-16T09:17:28.049" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:16:58.949" grpc.time_ms=5.887 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 215.093469] gitaly[1373]: time="2026-06-16T09:16:58.976Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-16T09:17:08.005" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:58.975" grpc.time_ms=0.202 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 215.101442] gitaly[1373]: time="2026-06-16T09:16:58.984Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-16T09:17:08.009" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-06-16T09:16:58.979" grpc.time_ms=4.837 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 215.124448] gitaly[1373]: time="2026-06-16T09:16:59.007Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-16T09:17:09.000" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:16:59.001" grpc.time_ms=5.877 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 215.136656] gitaly[1373]: time="2026-06-16T09:16:59.019Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-16T09:17:09.012" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:16:59.013" grpc.time_ms=6.315 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 215.157946] gitaly[1373]: time="2026-06-16T09:16:59.040Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=969 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-16T09:17:29.030" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:16:59.030" grpc.time_ms=9.959 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit setup-gitlab.service, in 216.38 seconds) vm-test-run-nixbot-gitlab> gitlab: waiting for unit nixbot.service vm-test-run-nixbot-gitlab> gitlab # [ 215.475605] gitaly[1373]: time="2026-06-16T09:16:59.358Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-16T09:17:09.005" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:16:59.356" grpc.time_ms=1.353 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 215.511552] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:59.394Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"48a3f2e0ae96c8d99143cba7","created_at":"2026-06-16T09:16:58.540Z","trace_propagation_headers":{"sentry-trace":"cd52cffba58d4d60b42ee524ec7eb63d-639fd0f7170b44c9","baggage":"sentry-trace_id=cd52cffba58d4d60b42ee524ec7eb63d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGEPX2ZW25XMXT68M1K8B","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:16:58.566Z","job_size_bytes":185,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-48a3f2e0ae96c8d99143cba7: done: 0.824015 sec","job_status":"done","queue_duration_s":0.004118,"scheduling_latency_s":0.00417,"gitaly_calls":14,"gitaly_duration_s":0.258437,"redis_calls":196,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.049766,"redis_read_bytes":1442,"redis_write_bytes":19009,"redis_cache_calls":14,"redis_cache_duration_s":0.003985,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":28,"redis_feature_flag_duration_s":0.008137,"redis_feature_flag_read_bytes":1044,"redis_feature_flag_write_bytes":3487,"redis_queues_calls":8,"redis_queues_duration_s":0.001507,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4934,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002571,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":91,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.0197,"redis_repository_cache_read_bytes":230,"redis_repository_cache_write_bytes":6310,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.013866,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.224,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.247,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.467055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000898717,"exclusive_lock_hold_duration_s":0.026902023,"duration_s":0.824015,"completed_at":"2026-06-16T09:16:59.394Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.067302,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 215.599589] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:59.396Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"5bd30cef0f01e39c89b135e9","created_at":"2026-06-16T09:16:58.698Z","trace_propagation_headers":{"sentry-trace":"cd52cffba58d4d60b42ee524ec7eb63d-f70fb6a48275488f","baggage":"sentry-trace_id=cd52cffba58d4d60b42ee524ec7eb63d,sentry-release=Unknown"},"correlation_id":"01KV7VGEPX2ZW25XMXT68M1K8B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:58.704Z","job_size_bytes":46,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-5bd30cef0f01e39c89b135e9: start","job_status":"start","queue_duration_s":0.692106,"scheduling_latency_s":0.692136} vm-test-run-nixbot-gitlab> gitlab # [ 215.601236] gitaly[1373]: time="2026-06-16T09:16:59.446Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-06-16T15:16:59.442" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:16:59.443" grpc.time_ms=2.467 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 215.602320] gitaly[1373]: time="2026-06-16T09:16:59.461Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=648 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-16T09:17:29.452" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:16:59.453" grpc.time_ms=8.068 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 215.611634] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:59.494Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"5bd30cef0f01e39c89b135e9","created_at":"2026-06-16T09:16:58.698Z","trace_propagation_headers":{"sentry-trace":"cd52cffba58d4d60b42ee524ec7eb63d-f70fb6a48275488f","baggage":"sentry-trace_id=cd52cffba58d4d60b42ee524ec7eb63d,sentry-release=Unknown"},"correlation_id":"01KV7VGEPX2ZW25XMXT68M1K8B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:58.704Z","job_size_bytes":46,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-5bd30cef0f01e39c89b135e9: done: 0.097717 sec","job_status":"done","queue_duration_s":0.692106,"scheduling_latency_s":0.692136,"gitaly_calls":2,"gitaly_duration_s":0.015641,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.011933,"redis_read_bytes":68,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001375,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000635,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.005713,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002789,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":752,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001421,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001848279,"duration_s":0.097717,"completed_at":"2026-06-16T09:16:59.494Z","load_balancing_strategy":"primary","db_duration_s":0.011349,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 215.691422] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:59.497Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"1db557aad4252a82b54c3fe7","created_at":"2026-06-16T09:16:58.719Z","trace_propagation_headers":{"sentry-trace":"cd52cffba58d4d60b42ee524ec7eb63d-f70fb6a48275488f","baggage":"sentry-trace_id=cd52cffba58d4d60b42ee524ec7eb63d,sentry-release=Unknown"},"correlation_id":"01KV7VGEPX2ZW25XMXT68M1K8B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:58.723Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-1db557aad4252a82b54c3fe7: start","job_status":"start","queue_duration_s":0.773437,"scheduling_latency_s":0.773477} vm-test-run-nixbot-gitlab> gitlab # [ 215.814111] gitaly[1373]: time="2026-06-16T09:16:59.697Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B diskcache=f4c8bce2-2b1b-4d67-8229-6e881f92e1cb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-16T15:16:59.521" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:16:59.522" pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 215.831385] gitaly[1373]: time="2026-06-16T09:16:59.714Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1283 command.oublock=0 command.real_time_ms=172 command.spawn_token_fork_ms=150 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-16T15:16:59.521" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:16:59.522" grpc.time_ms=174.96 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 215.879406] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:59.761Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"1db557aad4252a82b54c3fe7","created_at":"2026-06-16T09:16:58.719Z","trace_propagation_headers":{"sentry-trace":"cd52cffba58d4d60b42ee524ec7eb63d-f70fb6a48275488f","baggage":"sentry-trace_id=cd52cffba58d4d60b42ee524ec7eb63d,sentry-release=Unknown"},"correlation_id":"01KV7VGEPX2ZW25XMXT68M1K8B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:58.723Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-1db557aad4252a82b54c3fe7: done: 0.2648 sec","job_status":"done","queue_duration_s":0.773437,"scheduling_latency_s":0.773477,"gitaly_calls":1,"gitaly_duration_s":0.195946,"redis_calls":15,"redis_duration_s":0.004914,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000448,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002083,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000701,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001682,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.014,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.022,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.235927471,"duration_s":0.2648,"completed_at":"2026-06-16T09:16:59.761Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016765,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 215.922233] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:59.764Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"8b0633a2f493be0d783a31e6","created_at":"2026-06-16T09:16:58.863Z","trace_propagation_headers":{"sentry-trace":"cd52cffba58d4d60b42ee524ec7eb63d-f70fb6a48275488f","baggage":"sentry-trace_id=cd52cffba58d4d60b42ee524ec7eb63d,sentry-release=Unknown"},"correlation_id":"01KV7VGEPX2ZW25XMXT68M1K8B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:58.865Z","job_size_bytes":23,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8b0633a2f493be0d783a31e6: start","job_status":"start","queue_duration_s":0.898402,"scheduling_latency_s":0.898428} vm-test-run-nixbot-gitlab> gitlab # [ 215.924389] gitaly[1373]: time="2026-06-16T09:16:59.807Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=654 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-06-16T09:17:29.794" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:16:59.795" grpc.time_ms=11.448 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 216.005353] gitaly[1373]: time="2026-06-16T09:16:59.861Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:29.860" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:16:59.861" grpc.time_ms=0.622 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 216.006843] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:59.820Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"8b0633a2f493be0d783a31e6","created_at":"2026-06-16T09:16:58.863Z","trace_propagation_headers":{"sentry-trace":"cd52cffba58d4d60b42ee524ec7eb63d-f70fb6a48275488f","baggage":"sentry-trace_id=cd52cffba58d4d60b42ee524ec7eb63d,sentry-release=Unknown"},"correlation_id":"01KV7VGEPX2ZW25XMXT68M1K8B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:58.865Z","job_size_bytes":23,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-8b0633a2f493be0d783a31e6: done: 0.0567 sec","job_status":"done","queue_duration_s":0.898402,"scheduling_latency_s":0.898428,"gitaly_calls":1,"gitaly_duration_s":0.016124,"redis_calls":16,"redis_duration_s":0.007063,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000389,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002018,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.003482,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0567,"completed_at":"2026-06-16T09:16:59.820Z","load_balancing_strategy":"primary","db_duration_s":0.004339,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 216.029957] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:16:59.822Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"208d2a365786d60581a539fd","created_at":"2026-06-16T09:16:59.320Z","trace_propagation_headers":{"sentry-trace":"cd52cffba58d4d60b42ee524ec7eb63d-f70fb6a48275488f","baggage":"sentry-trace_id=cd52cffba58d4d60b42ee524ec7eb63d,sentry-release=Unknown"},"correlation_id":"01KV7VGEPX2ZW25XMXT68M1K8B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c9daf17241a7eda7fe346f1a2e8a2b66da21313a1d56cf14a54f048ee190deb","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:59.328Z","job_size_bytes":467,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-208d2a365786d60581a539fd: start","job_status":"start","queue_duration_s":0.494421,"scheduling_latency_s":0.494456} vm-test-run-nixbot-gitlab> gitlab # [ 216.057471] gitaly[1373]: time="2026-06-16T09:16:59.871Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-16T09:17:09.003" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:16:59.863" grpc.time_ms=7.229 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 216.144386] gitaly[1373]: time="2026-06-16T09:16:59.961Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-16T09:17:29.059" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:16:59.959" grpc.time_ms=1.429 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 216.145834] gitaly[1373]: time="2026-06-16T09:16:59.968Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=109 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-16T09:17:29.064" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:16:59.964" grpc.time_ms=3.701 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 216.190235] gitaly[1373]: time="2026-06-16T09:17:00.073Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-16T09:17:10.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:00.065" grpc.time_ms=7.16 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 216.235971] nixbot[3228]: {"ts": "2026-06-16T09:17:00.118Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} vm-test-run-nixbot-gitlab> gitlab # [ 216.257127] nixbot[3228]: {"ts": "2026-06-16T09:17:00.139Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} vm-test-run-nixbot-gitlab> gitlab # [ 216.271728] nixbot[3228]: {"ts": "2026-06-16T09:17:00.154Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 216.276278] nixbot[3228]: {"ts": "2026-06-16T09:17:00.159Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} vm-test-run-nixbot-gitlab> gitlab # [ 216.283208] nixbot[3228]: {"ts": "2026-06-16T09:17:00.166Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} vm-test-run-nixbot-gitlab> gitlab # [ 216.292431] nixbot[3228]: {"ts": "2026-06-16T09:17:00.175Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} vm-test-run-nixbot-gitlab> gitlab # [ 216.299637] nixbot[3228]: {"ts": "2026-06-16T09:17:00.182Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} vm-test-run-nixbot-gitlab> gitlab # [ 216.305511] nixbot[3228]: {"ts": "2026-06-16T09:17:00.188Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} vm-test-run-nixbot-gitlab> gitlab # [ 216.313875] nixbot[3228]: {"ts": "2026-06-16T09:17:00.196Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} vm-test-run-nixbot-gitlab> gitlab # [ 216.319628] nixbot[3228]: {"ts": "2026-06-16T09:17:00.202Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} vm-test-run-nixbot-gitlab> gitlab # [ 216.328477] nixbot[3228]: {"ts": "2026-06-16T09:17:00.211Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} vm-test-run-nixbot-gitlab> gitlab # [ 216.340199] nixbot[3228]: {"ts": "2026-06-16T09:17:00.222Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} vm-test-run-nixbot-gitlab> gitlab # [ 216.348445] nixbot[3228]: {"ts": "2026-06-16T09:17:00.231Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} vm-test-run-nixbot-gitlab> gitlab # [ 216.355164] nixbot[3228]: {"ts": "2026-06-16T09:17:00.237Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} vm-test-run-nixbot-gitlab> gitlab # [ 216.364168] nixbot[3228]: {"ts": "2026-06-16T09:17:00.246Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} vm-test-run-nixbot-gitlab> gitlab # [ 216.370693] nixbot[3228]: {"ts": "2026-06-16T09:17:00.253Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} vm-test-run-nixbot-gitlab> gitlab # [ 216.377518] nixbot[3228]: {"ts": "2026-06-16T09:17:00.260Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} vm-test-run-nixbot-gitlab> gitlab # [ 216.384245] nixbot[3228]: {"ts": "2026-06-16T09:17:00.266Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} vm-test-run-nixbot-gitlab> gitlab # [ 216.698330] nixbot[3228]: INFO: Started server process [3228] vm-test-run-nixbot-gitlab> gitlab # [ 216.701277] nixbot[3228]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) vm-test-run-nixbot-gitlab> gitlab # [ 216.704242] nixbot[3228]: INFO: ::ffff:127.0.0.1:51750 - "GET /health HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 216.712521] systemd[1]: Started nixbot CI service. vm-test-run-nixbot-gitlab> gitlab # [ 216.718295] systemd[1]: Reached target Multi-User System. vm-test-run-nixbot-gitlab> gitlab # [ 216.722646] systemd[1]: Startup finished in 1.556s (kernel) + 4.301s (initrd) + 3min 30.863s (userspace) = 3min 36.721s. vm-test-run-nixbot-gitlab> gitlab # [ 216.824450] gitaly[1373]: time="2026-06-16T09:17:00.706Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=966 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-16T09:17:30.089" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:00.690" grpc.time_ms=15.787 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 216.883877] gitaly[1373]: time="2026-06-16T09:17:00.766Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGEPX2ZW25XMXT68M1K8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-16T09:17:30.156" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:00.756" grpc.time_ms=10.23 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 216.927271] sidekiq-cluster[2167]: {"severity":"WARN","time":"2026-06-16T09:17:00.809Z","class":"CreatePipelineWorker","correlation_id":"01KV7VGEPX2ZW25XMXT68M1K8B","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf","ref":"refs/heads/master"},"retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 216.935505] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:00.817Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"208d2a365786d60581a539fd","created_at":"2026-06-16T09:16:59.320Z","trace_propagation_headers":{"sentry-trace":"cd52cffba58d4d60b42ee524ec7eb63d-f70fb6a48275488f","baggage":"sentry-trace_id=cd52cffba58d4d60b42ee524ec7eb63d,sentry-release=Unknown"},"correlation_id":"01KV7VGEPX2ZW25XMXT68M1K8B","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c9daf17241a7eda7fe346f1a2e8a2b66da21313a1d56cf14a54f048ee190deb","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:59.328Z","job_size_bytes":467,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-208d2a365786d60581a539fd: done: 0.99515 sec","job_status":"done","queue_duration_s":0.494421,"scheduling_latency_s":0.494456,"gitaly_calls":7,"gitaly_duration_s":0.113006,"redis_calls":53,"redis_duration_s":0.029258,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.001066,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.019059,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001516,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.006401,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001216,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":40,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":40,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.048,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.757549,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000599239,"duration_s":0.99515,"completed_at":"2026-06-16T09:17:00.817Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.081338,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 217.012579] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:00.819Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"eaa63f33f596ac91884bb4b1","created_at":"2026-06-16T09:16:59.470Z","trace_propagation_headers":{"sentry-trace":"cd52cffba58d4d60b42ee524ec7eb63d-83e9492ae96d4d13","baggage":"sentry-trace_id=cd52cffba58d4d60b42ee524ec7eb63d,sentry-release=Unknown"},"correlation_id":"01KV7VGEPX2ZW25XMXT68M1K8B","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:59.475Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-eaa63f33f596ac91884bb4b1: start","job_status":"start","queue_duration_s":1.344418,"scheduling_latency_s":1.344444} vm-test-run-nixbot-gitlab> gitlab # [ 217.015898] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:00.862Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"eaa63f33f596ac91884bb4b1","created_at":"2026-06-16T09:16:59.470Z","trace_propagation_headers":{"sentry-trace":"cd52cffba58d4d60b42ee524ec7eb63d-83e9492ae96d4d13","baggage":"sentry-trace_id=cd52cffba58d4d60b42ee524ec7eb63d,sentry-release=Unknown"},"correlation_id":"01KV7VGEPX2ZW25XMXT68M1K8B","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-06-16T09:16:59.475Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-eaa63f33f596ac91884bb4b1: done: 0.042651 sec","job_status":"done","queue_duration_s":1.344418,"scheduling_latency_s":1.344444,"redis_calls":19,"redis_duration_s":0.005208,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000932,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000872,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00251,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000894,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028253,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0075663250000000005,"duration_s":0.042651,"completed_at":"2026-06-16T09:17:00.862Z","load_balancing_strategy":"primary","db_duration_s":0.006248,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 217.044161] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:00.864Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"17a1a082b41dc3521e97df65","created_at":"2026-06-16T09:17:00.849Z","trace_propagation_headers":{"sentry-trace":"cd52cffba58d4d60b42ee524ec7eb63d-76b1c0011c614e27","baggage":"sentry-trace_id=cd52cffba58d4d60b42ee524ec7eb63d,sentry-release=Unknown"},"correlation_id":"01KV7VGEPX2ZW25XMXT68M1K8B","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:00.854Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-17a1a082b41dc3521e97df65: start","job_status":"start","queue_duration_s":0.010126,"scheduling_latency_s":0.010148} vm-test-run-nixbot-gitlab> gitlab # [ 217.071944] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:00.946Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"17a1a082b41dc3521e97df65","created_at":"2026-06-16T09:17:00.849Z","trace_propagation_headers":{"sentry-trace":"cd52cffba58d4d60b42ee524ec7eb63d-76b1c0011c614e27","baggage":"sentry-trace_id=cd52cffba58d4d60b42ee524ec7eb63d,sentry-release=Unknown"},"correlation_id":"01KV7VGEPX2ZW25XMXT68M1K8B","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:00.854Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-17a1a082b41dc3521e97df65: done: 0.082193 sec","job_status":"done","queue_duration_s":0.010126,"scheduling_latency_s":0.010148,"redis_calls":11,"redis_duration_s":0.0035900000000000003,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000867,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002063,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00066,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000416254,"duration_s":0.082193,"completed_at":"2026-06-16T09:17:00.946Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028992,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 217.441501] gitaly[1373]: time="2026-06-16T09:17:01.323Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=959 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGH9ADNEX23RXCZ4CH74M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-16T09:17:31.000" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:01.301" grpc.time_ms=21.818 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for unit nixbot.service, in 2.24 seconds) vm-test-run-nixbot-gitlab> gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health vm-test-run-nixbot-gitlab> gitlab # [ 217.790120] nixbot[3228]: INFO: ::ffff:127.0.0.1:51760 - "GET /health HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.05 seconds) vm-test-run-nixbot-gitlab> subtest: project discovered and webhook registered vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks vm-test-run-nixbot-gitlab> gitlab # [ 217.911488] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:01 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 71 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.12 seconds) vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> gitlab # [ 218.047494] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:01 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 1343 vm-test-run-nixbot-gitlab> gitlab # [ 218.051606] nixbot[3228]: {"ts": "2026-06-16T09:17:01.932Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 218.084448] nixbot[3228]: {"ts": "2026-06-16T09:17:01.967Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} vm-test-run-nixbot-gitlab> gitlab # [ 218.157740] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 58 vm-test-run-nixbot-gitlab> gitlab # [ 218.160084] nixbot[3228]: {"ts": "2026-06-16T09:17:02.042Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 218.163107] nixbot[3228]: {"ts": "2026-06-16T09:17:02.045Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} vm-test-run-nixbot-gitlab> gitlab # [ 218.325117] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 157 vm-test-run-nixbot-gitlab> gitlab # [ 218.327913] nixbot[3228]: {"ts": "2026-06-16T09:17:02.208Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 218.536405] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:02 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 192 vm-test-run-nixbot-gitlab> gitlab # [ 218.538492] nixbot[3228]: {"ts": "2026-06-16T09:17:02.420Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 218.675446] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 118 vm-test-run-nixbot-gitlab> gitlab # [ 218.685252] nixbot[3228]: {"ts": "2026-06-16T09:17:02.567Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 218.764447] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:02 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 63 vm-test-run-nixbot-gitlab> gitlab # [ 218.767917] nixbot[3228]: {"ts": "2026-06-16T09:17:02.648Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks vm-test-run-nixbot-gitlab> gitlab # [ 218.881612] gitaly[1373]: time="2026-06-16T09:17:02.763Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGKAF48DY372PWG64SPZG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-06-16T09:17:32.546" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:02.747" grpc.time_ms=15.466 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 218.953458] gitaly[1373]: time="2026-06-16T09:17:02.835Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGKAF48DY372PWG64SPZG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:32.633" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:02.834" grpc.time_ms=1.25 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 218.991475] gitaly[1373]: time="2026-06-16T09:17:02.874Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGKAF48DY372PWG64SPZG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-16T09:17:12.006" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:02.867" grpc.time_ms=6.81 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 219.027973] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 63 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.11 seconds) vm-test-run-nixbot-gitlab> [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-06-16T09:17:02.190Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] vm-test-run-nixbot-gitlab> (finished: subtest: project discovered and webhook registered, in 1.24 seconds) vm-test-run-nixbot-gitlab> subtest: push delivers the webhook vm-test-run-nixbot-gitlab> gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master vm-test-run-nixbot-gitlab> gitlab # [ 219.081520] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 293 vm-test-run-nixbot-gitlab> gitlab # [ 219.088531] nixbot[3228]: {"ts": "2026-06-16T09:17:02.970Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 219.104519] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:02 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 36 vm-test-run-nixbot-gitlab> gitlab # [ 219.178877] gitaly[1373]: time="2026-06-16T09:17:03.061Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=8 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKME9ETX92SVXYB0GDNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:03.047" grpc.time_ms=13.916 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 219.182199] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:03 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 200 "" "git/2.54.0" 74 vm-test-run-nixbot-gitlab> gitlab # Enumerating objects: 5, done. vm-test-run-nixbot-gitlab> gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. vm-test-run-nixbot-gitlab> gitlab # Delta compression using up to 4 threads vm-test-run-nixbot-gitlab> gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. vm-test-run-nixbot-gitlab> gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done. vm-test-run-nixbot-gitlab> gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) vm-test-run-nixbot-gitlab> gitlab # [ 219.264865] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:03 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 173 vm-test-run-nixbot-gitlab> gitlab # [ 219.269563] nixbot[3228]: {"ts": "2026-06-16T09:17:03.148Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} vm-test-run-nixbot-gitlab> gitlab # [ 219.274473] nixbot[3228]: {"ts": "2026-06-16T09:17:03.152Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} vm-test-run-nixbot-gitlab> gitlab # [ 219.379926] gitaly[1373]: time="2026-06-16T09:17:03.262Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKTH7SJDDJBJWEC4QX99 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:03.248" grpc.time_ms=13.796 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 219.381305] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 76 vm-test-run-nixbot-gitlab> gitlab # [ 219.474174] gitaly[1373]: time="2026-06-16T09:17:03.355Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-06-16T09:17:13.006" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:03.337" grpc.time_ms=17.287 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 219.496116] gitaly[1373]: time="2026-06-16T09:17:03.378Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGKX3Q1JR7HA7HWJQG0QG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:03.368" pid=1373 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 219.497847] gitaly[1373]: time="2026-06-16T09:17:03.378Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGKX3Q1JR7HA7HWJQG0QG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:03.368" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 219.501214] gitlab-workhorse[1288]: time="2026-06-16T09:17:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=149 vm-test-run-nixbot-gitlab> gitlab # [ 219.503124] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 112 vm-test-run-nixbot-gitlab> gitlab # [ 219.506188] gitaly[1373]: time="2026-06-16T09:17:03.379Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGKX3Q1JR7HA7HWJQG0QG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:03.368" grpc.time_ms=10.579 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 219.576740] gitaly[1373]: time="2026-06-16T09:17:03.459Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGM0QRQK23N7GJEWZP3QA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:03.440" pid=1373 remote_ip="2001:db8:1::1" request_sha=29b6bcc78e992767caac94190da59a7e17d64f5b response_bytes=377 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 219.578047] gitaly[1373]: time="2026-06-16T09:17:03.459Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1237 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGM0QRQK23N7GJEWZP3QA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:03.440" negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 219.581337] gitaly[1373]: time="2026-06-16T09:17:03.462Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1237 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGM0QRQK23N7GJEWZP3QA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=9 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:03.440" grpc.time_ms=21.372 negotiation.command=fetch negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 219.582858] gitlab-workhorse[1288]: time="2026-06-16T09:17:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=377 vm-test-run-nixbot-gitlab> gitlab # [ 219.583703] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 377 "" "git/2.54.0" 78 vm-test-run-nixbot-gitlab> gitlab # [ 219.697221] gitaly[1373]: time="2026-06-16T09:17:03.580Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-06-16T09:17:33.269" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-06-16T09:17:03.570" grpc.time_ms=9.503 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 219.734473] gitaly[1373]: time="2026-06-16T09:17:03.617Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 duration_ms=367ns method=POST pid=1373 status=200 url="http://unix/api/v4/internal/allowed" vm-test-run-nixbot-gitlab> gitlab # [ 219.750911] gitaly[1373]: time="2026-06-16T09:17:03.633Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGM6CK5X0S24DSZQ6G40J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=194 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:03.624" grpc.time_ms=9.025 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 219.753238] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 70 vm-test-run-nixbot-gitlab> gitlab # [ 219.785485] gitaly[1373]: time="2026-06-16T09:17:03.668Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 duration_ms=45ns method=POST pid=1373 status=200 url="http://unix/api/v4/internal/pre_receive" vm-test-run-nixbot-gitlab> gitlab # [ 219.788892] gitaly[1373]: time="2026-06-16T09:17:03.669Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:03.249" grpc.time_ms=419.319 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 219.836690] gitaly[1373]: time="2026-06-16T09:17:03.719Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:03.715" grpc.time_ms=3.207 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 219.857645] gitaly[1373]: time="2026-06-16T09:17:03.740Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGM8PGWND07562B73NSQD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:03.718" pid=1373 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 219.860330] gitaly[1373]: time="2026-06-16T09:17:03.741Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=5 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGM8PGWND07562B73NSQD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:03.718" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 219.862080] gitaly[1373]: time="2026-06-16T09:17:03.742Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=5 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGM8PGWND07562B73NSQD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:03.718" grpc.time_ms=23.473 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 219.863494] gitlab-workhorse[1288]: time="2026-06-16T09:17:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 219.864283] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 104 vm-test-run-nixbot-gitlab> gitlab # [ 219.937067] gitaly[1373]: time="2026-06-16T09:17:03.818Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:03.817" grpc.time_ms=0.549 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 219.977434] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:03.859Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"dbe11181dc294141b424012f","created_at":"2026-06-16T09:17:03.840Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b22d26b32325691b743e63637f0ff78","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:03.853Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-dbe11181dc294141b424012f: start","job_status":"start","queue_duration_s":0.006051,"scheduling_latency_s":0.006081} vm-test-run-nixbot-gitlab> gitlab # [ 219.986379] gitaly[1373]: time="2026-06-16T09:17:03.869Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:03.868" grpc.time_ms=0.724 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.038659] gitaly[1373]: time="2026-06-16T09:17:03.921Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGMF5BNRJZGXMK1Q8F294 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:33.618" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:03.919" grpc.time_ms=1.608 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.097583] gitaly[1373]: time="2026-06-16T09:17:03.980Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGMF5BNRJZGXMK1Q8F294 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-16T09:17:33.671" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:03.971" grpc.time_ms=8.373 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.118377] gitaly[1373]: time="2026-06-16T09:17:04.001Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGMF5BNRJZGXMK1Q8F294 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-16T09:17:13.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:03.994" grpc.time_ms=6.327 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.147129] gitaly[1373]: time="2026-06-16T09:17:04.029Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 duration_ms=77ns method=POST pid=1373 status=200 url="http://unix/api/v4/internal/post_receive" vm-test-run-nixbot-gitlab> gitlab # [ 220.152831] gitaly[1373]: time="2026-06-16T09:17:04.035Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:03.950" grpc.time_ms=84.512 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.182329] gitaly[1373]: time="2026-06-16T09:17:04.065Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGMF5BNRJZGXMK1Q8F294 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-16T09:17:14.004" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:17:04.064" grpc.time_ms=0.229 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.201120] gitaly[1373]: time="2026-06-16T09:17:04.083Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 diskcache=40212467-ae1a-46e0-b579-40781881259f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:03.187" pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.202302] gitaly[1373]: time="2026-06-16T09:17:04.083Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=186 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=13287 command.oublock=32 command.real_time_ms=867 command.spawn_token_fork_ms=2 command.system_time_ms=142 command.user_time_ms=44 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:03.187" grpc.time_ms=896.261 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.212080] gitaly[1373]: time="2026-06-16T09:17:04.095Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGMF5BNRJZGXMK1Q8F294 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-16T09:17:14.022" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:04.082" grpc.time_ms=11.925 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.262893] gitlab-workhorse[1288]: time="2026-06-16T09:17:04Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 vm-test-run-nixbot-gitlab> gitlab # [ 220.263994] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:04 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 1024 vm-test-run-nixbot-gitlab> gitlab # [ 220.264889] gitaly[1373]: time="2026-06-16T09:17:04.145Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGMF5BNRJZGXMK1Q8F294 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:04.144" grpc.time_ms=0.427 pid=1373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # To http://gitlab/root/test-flake.git vm-test-run-nixbot-gitlab> gitlab # ed9f2d2..f236896 master -> master vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.27 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: git -C /tmp/test-flake rev-parse master vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f236896cb5377ee51df845bb2a34d4215b03d088' vm-test-run-nixbot-gitlab> gitlab # [ 220.311129] gitaly[1373]: time="2026-06-16T09:17:04.193Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGMF5BNRJZGXMK1Q8F294 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:04.193" grpc.time_ms=0.408 pid=1373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 220.336268] gitaly[1373]: time="2026-06-16T09:17:04.219Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGMF5BNRJZGXMK1Q8F294 diskcache=3b2701ca-bf75-4d20-b613-a2d6e921411f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-16T09:17:14.036" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:17:04.097" pid=1373 ref=refs/keep-around/ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf remote_ip="2001:db8:1::1" revision=ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.337445] gitaly[1373]: time="2026-06-16T09:17:04.219Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=70 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5207 command.oublock=8 command.real_time_ms=112 command.spawn_token_fork_ms=5 command.system_time_ms=53 command.user_time_ms=17 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGMF5BNRJZGXMK1Q8F294 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-16T09:17:14.036" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:17:04.097" grpc.time_ms=122.093 pid=1373 ref=refs/keep-around/ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf remote_ip="2001:db8:1::1" revision=ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.379375] gitaly[1373]: time="2026-06-16T09:17:04.262Z" level=info msg="finished unary call with code OK" catfile.duration_ms=37 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=37 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGMF5BNRJZGXMK1Q8F294 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:34.023" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:04.223" grpc.time_ms=38.157 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.407366] nixbot[3228]: INFO: ::ffff:127.0.0.1:51776 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f236896cb5377ee51df845bb2a34d4215b03d088 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f236896cb5377ee51df845bb2a34d4215b03d088', in 0.10 seconds) vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> gitlab # [ 220.481976] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:04.364Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"dbe11181dc294141b424012f","created_at":"2026-06-16T09:17:03.840Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b22d26b32325691b743e63637f0ff78","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:03.853Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-dbe11181dc294141b424012f: done: 0.504851 sec","job_status":"done","queue_duration_s":0.006051,"scheduling_latency_s":0.006081,"redis_calls":13,"redis_duration_s":0.011762,"redis_read_bytes":11,"redis_write_bytes":1563,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00154,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004433,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005789,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":125,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":125,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.151,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.148,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.162477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000690311,"exclusive_lock_hold_duration_s":0.468659843,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.504851,"completed_at":"2026-06-16T09:17:04.364Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.327113,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 220.556426] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:04.438Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"26ac1799b780aaf52788cee3","created_at":"2026-06-16T09:17:03.861Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d61771f6a2b933b469f1fc6e1fd784b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:03.870Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-26ac1799b780aaf52788cee3: start","job_status":"start","queue_duration_s":0.568466,"scheduling_latency_s":0.568534} vm-test-run-nixbot-gitlab> gitlab # [ 220.617123] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:04.499Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"26ac1799b780aaf52788cee3","created_at":"2026-06-16T09:17:03.861Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d61771f6a2b933b469f1fc6e1fd784b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:03.870Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-26ac1799b780aaf52788cee3: done: 0.061121 sec","job_status":"done","queue_duration_s":0.568466,"scheduling_latency_s":0.568534,"redis_calls":18,"redis_duration_s":0.009167,"redis_read_bytes":20,"redis_write_bytes":4694,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002253,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000584,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.004823,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001507,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.061121,"completed_at":"2026-06-16T09:17:04.499Z","load_balancing_strategy":"primary","db_duration_s":0.003978,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 220.654744] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:04.501Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"bb3e8fa6741e9c3806b748bd","created_at":"2026-06-16T09:17:03.875Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bc0856db1257ca5285574dc550b15fdc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:03.954Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-bb3e8fa6741e9c3806b748bd: start","job_status":"start","queue_duration_s":0.547326,"scheduling_latency_s":0.547352} vm-test-run-nixbot-gitlab> gitlab # [ 220.670201] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:04.552Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 220.682739] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:04.564Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"bb3e8fa6741e9c3806b748bd","created_at":"2026-06-16T09:17:03.875Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bc0856db1257ca5285574dc550b15fdc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:03.954Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-bb3e8fa6741e9c3806b748bd: done: 0.063318 sec","job_status":"done","queue_duration_s":0.547326,"scheduling_latency_s":0.547352,"redis_calls":9,"redis_duration_s":0.033959,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000729,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002414,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.030816,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024252,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.063318,"completed_at":"2026-06-16T09:17:04.564Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000912,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 220.711540] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:04.567Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"50ffe99c261315aa419c2aca","created_at":"2026-06-16T09:17:03.964Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc3901de02d124909acd7020f1645137","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:03.975Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-50ffe99c261315aa419c2aca: start","job_status":"start","queue_duration_s":0.591636,"scheduling_latency_s":0.591687} vm-test-run-nixbot-gitlab> gitlab # [ 220.782673] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:04.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"50ffe99c261315aa419c2aca","created_at":"2026-06-16T09:17:03.964Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc3901de02d124909acd7020f1645137","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:03.975Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-50ffe99c261315aa419c2aca: done: 0.038939 sec","job_status":"done","queue_duration_s":0.591636,"scheduling_latency_s":0.591687,"redis_calls":13,"redis_duration_s":0.005461,"redis_read_bytes":11,"redis_write_bytes":1663,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002529,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":887,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00217,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000762,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038939,"completed_at":"2026-06-16T09:17:04.606Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002125,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 220.870075] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:04.608Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf f236896cb5377ee51df845bb2a34d4215b03d088 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"18987f731ad1280c59d1ce53","created_at":"2026-06-16T09:17:03.993Z","trace_propagation_headers":{"sentry-trace":"2d19cdc3dd144000aa4c9940b7686f4d-5a3ec193fae049d5","baggage":"sentry-trace_id=2d19cdc3dd144000aa4c9940b7686f4d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGKRGAKHYVHRQQQ4T7P98","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.001Z","job_size_bytes":185,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-18987f731ad1280c59d1ce53: start","job_status":"start","queue_duration_s":0.606791,"scheduling_latency_s":0.606824} vm-test-run-nixbot-gitlab> gitlab # [ 220.871344] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:04.622Z","message":"changes: \"ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf f236896cb5377ee51df845bb2a34d4215b03d088 refs/heads/master\\n\""} vm-test-run-nixbot-gitlab> gitlab # [ 220.872276] gitaly[1373]: time="2026-06-16T09:17:04.657Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-16T09:17:14.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:04.641" grpc.time_ms=15.036 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.873163] gitaly[1373]: time="2026-06-16T09:17:04.674Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-06-16T09:17:14.026" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:17:04.667" grpc.time_ms=7.322 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.942485] gitaly[1373]: time="2026-06-16T09:17:04.702Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-06-16T09:17:14.052" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:04.693" grpc.time_ms=8.748 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.943891] gitaly[1373]: time="2026-06-16T09:17:04.717Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-06-16T09:17:34.105" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:04.706" grpc.time_ms=10.677 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.944767] gitaly[1373]: time="2026-06-16T09:17:04.730Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=346 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-06-16T09:17:14.084" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-06-16T09:17:04.725" grpc.time_ms=4.806 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 220.945132] gitaly[1373]: time="2026-06-16T09:17:04.751Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-16T09:17:14.103" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:04.743" grpc.time_ms=7.621 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 221.025221] gitaly[1373]: time="2026-06-16T09:17:04.891Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-06-16T09:17:14.249" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:04.890" grpc.time_ms=1.498 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 221.027251] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:04.905Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf f236896cb5377ee51df845bb2a34d4215b03d088 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"18987f731ad1280c59d1ce53","created_at":"2026-06-16T09:17:03.993Z","trace_propagation_headers":{"sentry-trace":"2d19cdc3dd144000aa4c9940b7686f4d-5a3ec193fae049d5","baggage":"sentry-trace_id=2d19cdc3dd144000aa4c9940b7686f4d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGKRGAKHYVHRQQQ4T7P98","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.001Z","job_size_bytes":185,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-18987f731ad1280c59d1ce53: done: 0.297081 sec","job_status":"done","queue_duration_s":0.606791,"scheduling_latency_s":0.606824,"gitaly_calls":7,"gitaly_duration_s":0.077175,"redis_calls":112,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.024038,"redis_read_bytes":354,"redis_write_bytes":13069,"redis_cache_calls":9,"redis_cache_duration_s":0.002323,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000492,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.001313,"redis_queues_read_bytes":12,"redis_queues_write_bytes":6769,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.002422,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.010561,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.006927,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":29,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":29,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.016,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.188666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000803734,"exclusive_lock_hold_duration_s":0.030314188,"duration_s":0.297081,"completed_at":"2026-06-16T09:17:04.905Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024497,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 221.114421] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:04.907Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"23989c1490f76b6ec40d05d9","created_at":"2026-06-16T09:17:03.988Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e85be0d085121ebb087926314039140c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.010Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-23989c1490f76b6ec40d05d9: start","job_status":"start","queue_duration_s":0.896971,"scheduling_latency_s":0.896999} vm-test-run-nixbot-gitlab> gitlab # [ 221.116091] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:04.936Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"23989c1490f76b6ec40d05d9","created_at":"2026-06-16T09:17:03.988Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e85be0d085121ebb087926314039140c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.010Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-23989c1490f76b6ec40d05d9: done: 0.029641 sec","job_status":"done","queue_duration_s":0.896971,"scheduling_latency_s":0.896999,"redis_calls":11,"redis_duration_s":0.003027,"redis_read_bytes":9,"redis_write_bytes":1509,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001442,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000938946,"exclusive_lock_hold_duration_s":0.01278291,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.029641,"completed_at":"2026-06-16T09:17:04.936Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006852,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.206055] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:04.938Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"56b3a73ca59ee5aea3717ac4","created_at":"2026-06-16T09:17:04.046Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.053Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-56b3a73ca59ee5aea3717ac4: start","job_status":"start","queue_duration_s":0.885377,"scheduling_latency_s":0.885399} vm-test-run-nixbot-gitlab> gitlab # [ 221.207419] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.028Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"56b3a73ca59ee5aea3717ac4","created_at":"2026-06-16T09:17:04.046Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.053Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-56b3a73ca59ee5aea3717ac4: done: 0.090078 sec","job_status":"done","queue_duration_s":0.885377,"scheduling_latency_s":0.885399,"redis_calls":70,"redis_duration_s":0.016495,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002541,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000692,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.012569,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000693,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.073873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.090078,"completed_at":"2026-06-16T09:17:05.028Z","load_balancing_strategy":"primary","db_duration_s":0.001816,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.284443] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.029Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"7a710f39ddd94aac6895eb61","created_at":"2026-06-16T09:17:04.071Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de983fa2352580406bc6bc4b3449f757","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.089Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-7a710f39ddd94aac6895eb61: start","job_status":"start","queue_duration_s":0.940117,"scheduling_latency_s":0.940147} vm-test-run-nixbot-gitlab> gitlab # [ 221.285930] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.047Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"7a710f39ddd94aac6895eb61","created_at":"2026-06-16T09:17:04.071Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de983fa2352580406bc6bc4b3449f757","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.089Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-7a710f39ddd94aac6895eb61: done: 0.017953 sec","job_status":"done","queue_duration_s":0.940117,"scheduling_latency_s":0.940147,"redis_calls":11,"redis_duration_s":0.002572,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001152,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000495314,"exclusive_lock_hold_duration_s":0.002214527,"duration_s":0.017953,"completed_at":"2026-06-16T09:17:05.047Z","load_balancing_strategy":"primary","db_duration_s":0.001603,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f236896cb5377ee51df845bb2a34d4215b03d088' vm-test-run-nixbot-gitlab> gitlab # [ 221.365847] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.049Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"9d868def196bbc820836da18","created_at":"2026-06-16T09:17:04.092Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c81a67255db480b6d831fd393201ae8b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.098Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-9d868def196bbc820836da18: start","job_status":"start","queue_duration_s":0.951223,"scheduling_latency_s":0.951244} vm-test-run-nixbot-gitlab> gitlab # [ 221.367178] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.058Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 221.368110] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.063Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"9d868def196bbc820836da18","created_at":"2026-06-16T09:17:04.092Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c81a67255db480b6d831fd393201ae8b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.098Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-9d868def196bbc820836da18: done: 0.014083 sec","job_status":"done","queue_duration_s":0.951223,"scheduling_latency_s":0.951244,"redis_calls":9,"redis_duration_s":0.0022890000000000002,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000714,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001139,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014083,"completed_at":"2026-06-16T09:17:05.063Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000406,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.439274] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.064Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d1b760f43b0f9e862b37bd6e","created_at":"2026-06-16T09:17:04.101Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1352211c5e51a036eb5c71ab9ce84fe7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-16T09:17:04.108Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d1b760f43b0f9e862b37bd6e: start","job_status":"start","queue_duration_s":0.956485,"scheduling_latency_s":0.956506} vm-test-run-nixbot-gitlab> gitlab # [ 221.448950] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.074Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f236896cb5377ee51df845bb2a34d4215b03d088', in 0.14 seconds) vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> gitlab # [ 221.519318] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.078Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d1b760f43b0f9e862b37bd6e","created_at":"2026-06-16T09:17:04.101Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1352211c5e51a036eb5c71ab9ce84fe7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-06-16T09:17:04.108Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d1b760f43b0f9e862b37bd6e: done: 0.013865 sec","job_status":"done","queue_duration_s":0.956485,"scheduling_latency_s":0.956506,"redis_calls":9,"redis_duration_s":0.002038,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000418,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00119,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00043,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013865,"completed_at":"2026-06-16T09:17:05.078Z","load_balancing_strategy":"primary","db_duration_s":0.000374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.521116] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.079Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"e4d4f0ad66d4e0e22d339976","created_at":"2026-06-16T09:17:04.113Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"92699969fdbd5929c3a92031e33dc061","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.120Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-e4d4f0ad66d4e0e22d339976: start","job_status":"start","queue_duration_s":0.959406,"scheduling_latency_s":0.959428} vm-test-run-nixbot-gitlab> gitlab # [ 221.607776] nixbot[3228]: INFO: ::ffff:127.0.0.1:51788 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f236896cb5377ee51df845bb2a34d4215b03d088 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 221.636722] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:05 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 1673 vm-test-run-nixbot-gitlab> gitlab # [ 221.641975] nixbot[3228]: {"ts": "2026-06-16T09:17:05.524Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 221.650834] nixbot[3228]: {"ts": "2026-06-16T09:17:05.533Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-ed9f2d2dd314-58fca0ac", "argv0": "bwrap"} vm-test-run-nixbot-gitlab> gitlab # [ 221.726082] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.608Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"e4d4f0ad66d4e0e22d339976","created_at":"2026-06-16T09:17:04.113Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"92699969fdbd5929c3a92031e33dc061","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.120Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-e4d4f0ad66d4e0e22d339976: done: 0.528537 sec","job_status":"done","queue_duration_s":0.959406,"scheduling_latency_s":0.959428,"redis_calls":573,"redis_duration_s":0.14891600000000002,"redis_read_bytes":571,"redis_write_bytes":49790,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000458,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001245,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.147213,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.417953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.528537,"completed_at":"2026-06-16T09:17:05.608Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000428,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.731323] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.610Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8133fdc93f450ab50bf212f4","created_at":"2026-06-16T09:17:04.130Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39ccd562ccf22ee315b70dd97cde30e0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.137Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8133fdc93f450ab50bf212f4: start","job_status":"start","queue_duration_s":1.472872,"scheduling_latency_s":1.472906} vm-test-run-nixbot-gitlab> gitlab # [ 221.767932] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.650Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8133fdc93f450ab50bf212f4","created_at":"2026-06-16T09:17:04.130Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39ccd562ccf22ee315b70dd97cde30e0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.137Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8133fdc93f450ab50bf212f4: done: 0.040171 sec","job_status":"done","queue_duration_s":1.472872,"scheduling_latency_s":1.472906,"redis_calls":11,"redis_duration_s":0.0028550000000000003,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001221,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001185,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000600635,"exclusive_lock_hold_duration_s":0.013272078,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.040171,"completed_at":"2026-06-16T09:17:05.650Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01208,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.846329] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.652Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"9aae0a5802fd38c0488ec00b","created_at":"2026-06-16T09:17:04.140Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f7ced156f7095f2c398e3b8a56567ce","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.152Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-9aae0a5802fd38c0488ec00b: start","job_status":"start","queue_duration_s":1.499929,"scheduling_latency_s":1.499954} vm-test-run-nixbot-gitlab> gitlab # [ 221.847740] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.663Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 221.848933] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.668Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"9aae0a5802fd38c0488ec00b","created_at":"2026-06-16T09:17:04.140Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f7ced156f7095f2c398e3b8a56567ce","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.152Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-9aae0a5802fd38c0488ec00b: done: 0.016426 sec","job_status":"done","queue_duration_s":1.499929,"scheduling_latency_s":1.499954,"redis_calls":9,"redis_duration_s":0.001961,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000425,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001111,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000425,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016426,"completed_at":"2026-06-16T09:17:05.668Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000557,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 221.919785] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.670Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ArchiveTracesCronWorker","jid":"1b949724bfa99cdb514b1bd9","created_at":"2026-06-16T09:17:04.159Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7946a4a26b353e37839003a35af6da0a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f0234f524ae629146c97646d3c809b290751f669b415a8b01cafe8f68b01506d","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.163Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ArchiveTracesCronWorker JID-1b949724bfa99cdb514b1bd9: start","job_status":"start","queue_duration_s":1.506782,"scheduling_latency_s":1.506806} vm-test-run-nixbot-gitlab> gitlab # [ 221.989942] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.702Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ArchiveTracesCronWorker","jid":"1b949724bfa99cdb514b1bd9","created_at":"2026-06-16T09:17:04.159Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7946a4a26b353e37839003a35af6da0a","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f0234f524ae629146c97646d3c809b290751f669b415a8b01cafe8f68b01506d","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.163Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ArchiveTracesCronWorker JID-1b949724bfa99cdb514b1bd9: done: 0.032288 sec","job_status":"done","queue_duration_s":1.506782,"scheduling_latency_s":1.506806,"redis_calls":12,"redis_duration_s":0.003249,"redis_read_bytes":11,"redis_write_bytes":1354,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000423,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":297,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001443,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001383,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":519,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000871619,"exclusive_lock_hold_duration_s":0.009237206999999999,"duration_s":0.032288,"completed_at":"2026-06-16T09:17:05.702Z","load_balancing_strategy":"primary","db_duration_s":0.004612,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.065486] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.705Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f15328cc7b2da0118ffa8206","created_at":"2026-06-16T09:17:04.167Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"29e42612711de0a534718798966307f0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.172Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f15328cc7b2da0118ffa8206: start","job_status":"start","queue_duration_s":1.532984,"scheduling_latency_s":1.533035} vm-test-run-nixbot-gitlab> gitlab # [ 222.066889] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.727Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f15328cc7b2da0118ffa8206","created_at":"2026-06-16T09:17:04.167Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"29e42612711de0a534718798966307f0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.172Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f15328cc7b2da0118ffa8206: done: 0.022061 sec","job_status":"done","queue_duration_s":1.532984,"scheduling_latency_s":1.533035,"redis_calls":9,"redis_duration_s":0.0023239999999999997,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000493,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001176,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000655,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013351,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022061,"completed_at":"2026-06-16T09:17:05.727Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001321,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.089436] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.728Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"6b6b0cf4e03dac5287571414","created_at":"2026-06-16T09:17:04.175Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"826b694519a08c77ed250badb0c0f2b8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.185Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-6b6b0cf4e03dac5287571414: start","job_status":"start","queue_duration_s":1.543718,"scheduling_latency_s":1.543739} vm-test-run-nixbot-gitlab> gitlab # [ 222.113621] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.743Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"6b6b0cf4e03dac5287571414","created_at":"2026-06-16T09:17:04.175Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"826b694519a08c77ed250badb0c0f2b8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.185Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-6b6b0cf4e03dac5287571414: done: 0.014976 sec","job_status":"done","queue_duration_s":1.543718,"scheduling_latency_s":1.543739,"redis_calls":9,"redis_duration_s":0.002057,"redis_read_bytes":7,"redis_write_bytes":1192,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000474,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00115,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000433,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009533,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014976,"completed_at":"2026-06-16T09:17:05.743Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000469,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.187269] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.745Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"0caa4554e7ed00c0817ebe8e","created_at":"2026-06-16T09:17:04.210Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85fa8ab2098126cf3c8f2ea3b01dd976","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-16T09:17:04.222Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-0caa4554e7ed00c0817ebe8e: start","job_status":"start","queue_duration_s":1.52259,"scheduling_latency_s":1.522612} vm-test-run-nixbot-gitlab> gitlab # [ 222.202054] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.754Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 222.208554] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.759Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"0caa4554e7ed00c0817ebe8e","created_at":"2026-06-16T09:17:04.210Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85fa8ab2098126cf3c8f2ea3b01dd976","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-06-16T09:17:04.222Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-0caa4554e7ed00c0817ebe8e: done: 0.014231 sec","job_status":"done","queue_duration_s":1.52259,"scheduling_latency_s":1.522612,"redis_calls":9,"redis_duration_s":0.002159,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000475,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001241,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014231,"completed_at":"2026-06-16T09:17:05.759Z","load_balancing_strategy":"primary","db_duration_s":0.000409,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.287349] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.761Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"45be8632a727b2ca5e4faa3b","created_at":"2026-06-16T09:17:04.234Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d1c9da41236916655b8b7a46e318af12","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.244Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-45be8632a727b2ca5e4faa3b: start","job_status":"start","queue_duration_s":1.516776,"scheduling_latency_s":1.516835} vm-test-run-nixbot-gitlab> gitlab # [ 222.363411] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.776Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 222.364904] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.782Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"45be8632a727b2ca5e4faa3b","created_at":"2026-06-16T09:17:04.234Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d1c9da41236916655b8b7a46e318af12","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.244Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-45be8632a727b2ca5e4faa3b: done: 0.02085 sec","job_status":"done","queue_duration_s":1.516776,"scheduling_latency_s":1.516835,"redis_calls":9,"redis_duration_s":0.002092,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00045,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001186,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000456,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02085,"completed_at":"2026-06-16T09:17:05.782Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000556,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.436256] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.783Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"c10bb2a21309ed3994bfbc4a","created_at":"2026-06-16T09:17:04.254Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cc03e56e6fe0d2f7edc69984835cf0e","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.263Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-c10bb2a21309ed3994bfbc4a: start","job_status":"start","queue_duration_s":1.520298,"scheduling_latency_s":1.520321} vm-test-run-nixbot-gitlab> gitlab # [ 222.438074] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.801Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"c10bb2a21309ed3994bfbc4a","created_at":"2026-06-16T09:17:04.254Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cc03e56e6fe0d2f7edc69984835cf0e","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.263Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-c10bb2a21309ed3994bfbc4a: done: 0.017278 sec","job_status":"done","queue_duration_s":1.520298,"scheduling_latency_s":1.520321,"redis_calls":11,"redis_duration_s":0.00251,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001152,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000943,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010452,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000473524,"exclusive_lock_hold_duration_s":0.00193628,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.017278,"completed_at":"2026-06-16T09:17:05.801Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001447,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f236896cb5377ee51df845bb2a34d4215b03d088' vm-test-run-nixbot-gitlab> gitlab # [ 222.519170] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.803Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"11770a467246b6f259e40408","created_at":"2026-06-16T09:17:04.270Z","trace_propagation_headers":{"sentry-trace":"37688b26f51c473b96d8e2d67a015395-9973b03d6ce841f9","baggage":"sentry-trace_id=37688b26f51c473b96d8e2d67a015395,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGMF5BNRJZGXMK1Q8F294","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.280Z","job_size_bytes":65,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-11770a467246b6f259e40408: start","job_status":"start","queue_duration_s":1.522889,"scheduling_latency_s":1.52291} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f236896cb5377ee51df845bb2a34d4215b03d088', in 0.04 seconds) vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> gitlab # [ 222.520401] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.847Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"11770a467246b6f259e40408","created_at":"2026-06-16T09:17:04.270Z","trace_propagation_headers":{"sentry-trace":"37688b26f51c473b96d8e2d67a015395-9973b03d6ce841f9","baggage":"sentry-trace_id=37688b26f51c473b96d8e2d67a015395,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGMF5BNRJZGXMK1Q8F294","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.280Z","job_size_bytes":65,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-11770a467246b6f259e40408: done: 0.044111 sec","job_status":"done","queue_duration_s":1.522889,"scheduling_latency_s":1.52291,"redis_calls":18,"redis_duration_s":0.004142,"redis_read_bytes":271,"redis_write_bytes":2103,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002322,"redis_feature_flag_read_bytes":264,"redis_feature_flag_write_bytes":1285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001348,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000374349,"duration_s":0.044111,"completed_at":"2026-06-16T09:17:05.847Z","load_balancing_strategy":"primary","db_duration_s":0.008389,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 222.588795] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.848Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"285dc907b5b20f2ae755972f","created_at":"2026-06-16T09:17:04.272Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"154adf9a878bd8e909b3facee9da72ba","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.283Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-285dc907b5b20f2ae755972f: start","job_status":"start","queue_duration_s":1.565549,"scheduling_latency_s":1.56557} vm-test-run-nixbot-gitlab> gitlab # [ 222.590907] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.871Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"285dc907b5b20f2ae755972f","created_at":"2026-06-16T09:17:04.272Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"154adf9a878bd8e909b3facee9da72ba","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.283Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-285dc907b5b20f2ae755972f: done: 0.022724 sec","job_status":"done","queue_duration_s":1.565549,"scheduling_latency_s":1.56557,"redis_calls":9,"redis_duration_s":0.002299,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000466,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001371,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000462,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.022724,"completed_at":"2026-06-16T09:17:05.871Z","load_balancing_strategy":"primary","db_duration_s":0.004857,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.662225] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.873Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"ebe11e9088c29f4abdfb52c7","created_at":"2026-06-16T09:17:04.281Z","trace_propagation_headers":{"sentry-trace":"37688b26f51c473b96d8e2d67a015395-9973b03d6ce841f9","baggage":"sentry-trace_id=37688b26f51c473b96d8e2d67a015395,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGMF5BNRJZGXMK1Q8F294","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.285Z","job_size_bytes":65,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-ebe11e9088c29f4abdfb52c7: start","job_status":"start","queue_duration_s":1.587361,"scheduling_latency_s":1.587393} vm-test-run-nixbot-gitlab> gitlab # [ 222.674054] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.939Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"ebe11e9088c29f4abdfb52c7","created_at":"2026-06-16T09:17:04.281Z","trace_propagation_headers":{"sentry-trace":"37688b26f51c473b96d8e2d67a015395-9973b03d6ce841f9","baggage":"sentry-trace_id=37688b26f51c473b96d8e2d67a015395,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGMF5BNRJZGXMK1Q8F294","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.285Z","job_size_bytes":65,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-ebe11e9088c29f4abdfb52c7: done: 0.066547 sec","job_status":"done","queue_duration_s":1.587361,"scheduling_latency_s":1.587393,"redis_calls":21,"redis_duration_s":0.004936,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002134,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001398,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001404,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000366806,"duration_s":0.066547,"completed_at":"2026-06-16T09:17:05.939Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010986,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.809307] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.941Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"a1af651508a4a61bfe0a7667","created_at":"2026-06-16T09:17:04.294Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"380370700206bb7c366a5294ef7e1667","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.304Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-a1af651508a4a61bfe0a7667: start","job_status":"start","queue_duration_s":1.637138,"scheduling_latency_s":1.637179} vm-test-run-nixbot-gitlab> gitlab # [ 222.810777] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.965Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"a1af651508a4a61bfe0a7667","created_at":"2026-06-16T09:17:04.294Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"380370700206bb7c366a5294ef7e1667","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.304Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-a1af651508a4a61bfe0a7667: done: 0.023357 sec","job_status":"done","queue_duration_s":1.637138,"scheduling_latency_s":1.637179,"redis_calls":9,"redis_duration_s":0.0035570000000000003,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000477,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002592,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000488,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023357,"completed_at":"2026-06-16T09:17:05.965Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003245,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.836300] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.966Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"cdc02491f0b501b1f66c0d4d","created_at":"2026-06-16T09:17:04.324Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"052eadf77c4bacbc20833f19d701a443","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-16T09:17:04.336Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-cdc02491f0b501b1f66c0d4d: start","job_status":"start","queue_duration_s":1.629946,"scheduling_latency_s":1.629969} vm-test-run-nixbot-gitlab> gitlab # [ 222.857172] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.986Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"cdc02491f0b501b1f66c0d4d","created_at":"2026-06-16T09:17:04.324Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"052eadf77c4bacbc20833f19d701a443","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-06-16T09:17:04.336Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-cdc02491f0b501b1f66c0d4d: done: 0.019886 sec","job_status":"done","queue_duration_s":1.629946,"scheduling_latency_s":1.629969,"redis_calls":9,"redis_duration_s":0.002017,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000453,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000435,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019886,"completed_at":"2026-06-16T09:17:05.986Z","load_balancing_strategy":"primary","db_duration_s":0.002068,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 222.926179] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:05.988Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"db2de61b08abc066877dfcb4","created_at":"2026-06-16T09:17:04.352Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bccdd9dfe3bfd1a9aca42441d7edfe75","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.363Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-db2de61b08abc066877dfcb4: start","job_status":"start","queue_duration_s":1.62475,"scheduling_latency_s":1.624807} vm-test-run-nixbot-gitlab> gitlab # [ 222.944923] systemd[1]: Started Nix Daemon. vm-test-run-nixbot-gitlab> gitlab # [ 222.946545] nixbot[3228]: INFO: ::ffff:127.0.0.1:51804 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f236896cb5377ee51df845bb2a34d4215b03d088 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 222.948632] gitaly[1373]: time="2026-06-16T09:17:06.196Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGMF5BNRJZGXMK1Q8F294 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-16T09:17:36.094" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:06.194" grpc.time_ms=1.166 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 222.969481] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.011Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"db2de61b08abc066877dfcb4","created_at":"2026-06-16T09:17:04.352Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-3ce38dd576e74603","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bccdd9dfe3bfd1a9aca42441d7edfe75","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.363Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-db2de61b08abc066877dfcb4: done: 0.023295 sec","job_status":"done","queue_duration_s":1.62475,"scheduling_latency_s":1.624807,"redis_calls":12,"redis_duration_s":0.00363,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000461,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002166,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002668496,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.023295,"completed_at":"2026-06-16T09:17:06.011Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002304,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 223.056934] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.013Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3bc4b10ac7a51760f2507d71","created_at":"2026-06-16T09:17:04.478Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-02d8a32c3c2041e7","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1d61771f6a2b933b469f1fc6e1fd784b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.493Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3bc4b10ac7a51760f2507d71: start","job_status":"start","queue_duration_s":1.519303,"scheduling_latency_s":1.519335} vm-test-run-nixbot-gitlab> gitlab # [ 223.058183] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.028Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 223.060837] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.034Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3bc4b10ac7a51760f2507d71","created_at":"2026-06-16T09:17:04.478Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-02d8a32c3c2041e7","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1d61771f6a2b933b469f1fc6e1fd784b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.493Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3bc4b10ac7a51760f2507d71: done: 0.021565 sec","job_status":"done","queue_duration_s":1.519303,"scheduling_latency_s":1.519335,"redis_calls":12,"redis_duration_s":0.002859,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000571,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00136,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021565,"completed_at":"2026-06-16T09:17:06.034Z","load_balancing_strategy":"primary","db_duration_s":0.002065,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.139665] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.035Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"313ab1d43da1d4456d819baf","created_at":"2026-06-16T09:17:04.478Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-02d8a32c3c2041e7","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1d61771f6a2b933b469f1fc6e1fd784b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.493Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-313ab1d43da1d4456d819baf: start","job_status":"start","queue_duration_s":1.542145,"scheduling_latency_s":1.542176} vm-test-run-nixbot-gitlab> gitlab # [ 223.143839] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.045Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 223.146130] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.052Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"313ab1d43da1d4456d819baf","created_at":"2026-06-16T09:17:04.478Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-02d8a32c3c2041e7","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1d61771f6a2b933b469f1fc6e1fd784b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.493Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-313ab1d43da1d4456d819baf: done: 0.016511 sec","job_status":"done","queue_duration_s":1.542145,"scheduling_latency_s":1.542176,"redis_calls":10,"redis_duration_s":0.002439,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00108,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001359,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010348,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016511,"completed_at":"2026-06-16T09:17:06.052Z","load_balancing_strategy":"primary","db_duration_s":0.001621,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.224849] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.053Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0a946ece9a715fa3ad7aca43","created_at":"2026-06-16T09:17:04.478Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-02d8a32c3c2041e7","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1d61771f6a2b933b469f1fc6e1fd784b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.493Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0a946ece9a715fa3ad7aca43: start","job_status":"start","queue_duration_s":1.55993,"scheduling_latency_s":1.559954} vm-test-run-nixbot-gitlab> gitlab # [ 223.227048] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.063Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 223.229880] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.069Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0a946ece9a715fa3ad7aca43","created_at":"2026-06-16T09:17:04.478Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-02d8a32c3c2041e7","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1d61771f6a2b933b469f1fc6e1fd784b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.493Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0a946ece9a715fa3ad7aca43: done: 0.015794 sec","job_status":"done","queue_duration_s":1.55993,"scheduling_latency_s":1.559954,"redis_calls":10,"redis_duration_s":0.002255,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001305,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010237,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015794,"completed_at":"2026-06-16T09:17:06.069Z","load_balancing_strategy":"primary","db_duration_s":0.0017,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.315370] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.074Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"acbd84b9a6ab9adb93d0a1ca","created_at":"2026-06-16T09:17:04.478Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-02d8a32c3c2041e7","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1d61771f6a2b933b469f1fc6e1fd784b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.493Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-acbd84b9a6ab9adb93d0a1ca: start","job_status":"start","queue_duration_s":1.581008,"scheduling_latency_s":1.581032} vm-test-run-nixbot-gitlab> gitlab # [ 223.317472] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.091Z","message":"No existing merge request to be cleaned up."} vm-test-run-nixbot-gitlab> gitlab # [ 223.320148] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.101Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"acbd84b9a6ab9adb93d0a1ca","created_at":"2026-06-16T09:17:04.478Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-02d8a32c3c2041e7","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1d61771f6a2b933b469f1fc6e1fd784b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.493Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-acbd84b9a6ab9adb93d0a1ca: done: 0.027062 sec","job_status":"done","queue_duration_s":1.581008,"scheduling_latency_s":1.581032,"redis_calls":10,"redis_duration_s":0.005167000000000001,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001172,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003995,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027062,"completed_at":"2026-06-16T09:17:06.101Z","load_balancing_strategy":"primary","db_duration_s":0.007433,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.401244] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.103Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"adfd149918021f771d73388b","created_at":"2026-06-16T09:17:04.489Z","trace_propagation_headers":{"sentry-trace":"37688b26f51c473b96d8e2d67a015395-9973b03d6ce841f9","baggage":"sentry-trace_id=37688b26f51c473b96d8e2d67a015395,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGMF5BNRJZGXMK1Q8F294","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.502Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-adfd149918021f771d73388b: start","job_status":"start","queue_duration_s":1.60063,"scheduling_latency_s":1.600655} vm-test-run-nixbot-gitlab> gitlab # [ 223.410185] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.271Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"adfd149918021f771d73388b","created_at":"2026-06-16T09:17:04.489Z","trace_propagation_headers":{"sentry-trace":"37688b26f51c473b96d8e2d67a015395-9973b03d6ce841f9","baggage":"sentry-trace_id=37688b26f51c473b96d8e2d67a015395,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGMF5BNRJZGXMK1Q8F294","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.502Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-adfd149918021f771d73388b: done: 0.1684 sec","job_status":"done","queue_duration_s":1.60063,"scheduling_latency_s":1.600655,"gitaly_calls":3,"gitaly_duration_s":0.009519,"redis_calls":48,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.010407,"redis_read_bytes":209,"redis_write_bytes":6803,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001233,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":16,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001624,"redis_cache_read_bytes":32,"redis_cache_write_bytes":1470,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.001697,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000373,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.002124,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000593,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000238,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001292,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.004,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.007,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.121628,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.133445173,"duration_s":0.1684,"completed_at":"2026-06-16T09:17:06.271Z","load_balancing_strategy":"primary","db_duration_s":0.031086,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 223.503915] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.273Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"6fde9d2d3e478dd7d90dc1be","created_at":"2026-06-16T09:17:04.678Z","trace_propagation_headers":{"sentry-trace":"2d19cdc3dd144000aa4c9940b7686f4d-898a4ed89fe94159","baggage":"sentry-trace_id=2d19cdc3dd144000aa4c9940b7686f4d,sentry-release=Unknown"},"correlation_id":"01KV7VGKRGAKHYVHRQQQ4T7P98","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.682Z","job_size_bytes":46,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-6fde9d2d3e478dd7d90dc1be: start","job_status":"start","queue_duration_s":1.591409,"scheduling_latency_s":1.591433} vm-test-run-nixbot-gitlab> gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f236896cb5377ee51df845bb2a34d4215b03d088' vm-test-run-nixbot-gitlab> gitlab # [ 223.513432] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.293Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"6fde9d2d3e478dd7d90dc1be","created_at":"2026-06-16T09:17:04.678Z","trace_propagation_headers":{"sentry-trace":"2d19cdc3dd144000aa4c9940b7686f4d-898a4ed89fe94159","baggage":"sentry-trace_id=2d19cdc3dd144000aa4c9940b7686f4d,sentry-release=Unknown"},"correlation_id":"01KV7VGKRGAKHYVHRQQQ4T7P98","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.682Z","job_size_bytes":46,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-6fde9d2d3e478dd7d90dc1be: done: 0.019646 sec","job_status":"done","queue_duration_s":1.591409,"scheduling_latency_s":1.591433,"redis_calls":9,"redis_duration_s":0.002116,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0012,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.00024,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000676,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019646,"completed_at":"2026-06-16T09:17:06.293Z","load_balancing_strategy":"primary","db_duration_s":0.002169,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 223.582403] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.294Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"bb9c4fd0b7ba1ca98378ae9f","created_at":"2026-06-16T09:17:04.687Z","trace_propagation_headers":{"sentry-trace":"2d19cdc3dd144000aa4c9940b7686f4d-898a4ed89fe94159","baggage":"sentry-trace_id=2d19cdc3dd144000aa4c9940b7686f4d,sentry-release=Unknown"},"correlation_id":"01KV7VGKRGAKHYVHRQQQ4T7P98","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.689Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-bb9c4fd0b7ba1ca98378ae9f: start","job_status":"start","queue_duration_s":1.605056,"scheduling_latency_s":1.605079} vm-test-run-nixbot-gitlab> gitlab # [ 223.676457] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.360Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"bb9c4fd0b7ba1ca98378ae9f","created_at":"2026-06-16T09:17:04.687Z","trace_propagation_headers":{"sentry-trace":"2d19cdc3dd144000aa4c9940b7686f4d-898a4ed89fe94159","baggage":"sentry-trace_id=2d19cdc3dd144000aa4c9940b7686f4d,sentry-release=Unknown"},"correlation_id":"01KV7VGKRGAKHYVHRQQQ4T7P98","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.689Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-bb9c4fd0b7ba1ca98378ae9f: done: 0.065645 sec","job_status":"done","queue_duration_s":1.605056,"scheduling_latency_s":1.605079,"gitaly_calls":1,"gitaly_duration_s":0.033946,"redis_calls":12,"redis_duration_s":0.0031450000000000002,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001262,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000662,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001221,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.047652146,"duration_s":0.065645,"completed_at":"2026-06-16T09:17:06.360Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004093,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 223.680438] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:06.362Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"34622e6046e92a3090b20d8b","created_at":"2026-06-16T09:17:04.788Z","trace_propagation_headers":{"sentry-trace":"2d19cdc3dd144000aa4c9940b7686f4d-898a4ed89fe94159","baggage":"sentry-trace_id=2d19cdc3dd144000aa4c9940b7686f4d,sentry-release=Unknown"},"correlation_id":"01KV7VGKRGAKHYVHRQQQ4T7P98","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:935f70ed74e4002b458115b6d00f4313d33c954f54732e02826c2b70f53f80e5","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.790Z","job_size_bytes":467,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-34622e6046e92a3090b20d8b: start","job_status":"start","queue_duration_s":1.571325,"scheduling_latency_s":1.571349} vm-test-run-nixbot-gitlab> gitlab # [ 223.772071] gitaly[1373]: time="2026-06-16T09:17:06.228Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGMF5BNRJZGXMK1Q8F294 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:36.126" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:06.227" grpc.time_ms=0.858 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 223.774465] nix-daemon[3491]: accepted connection from pid 3488, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 223.775828] nix-daemon[3491]: accepted connection from pid 3490, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 223.777341] nixbot[3228]: INFO: ::ffff:127.0.0.1:56954 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted vm-test-run-nixbot-gitlab> gitlab # [ 223.779085] nixbot[3228]: {"ts": "2026-06-16T09:17:07.364Z", "level": "info", "logger": "nixbot.canceller", "msg": "cancelling build", "build_id": 1, "reason": "superseded"} vm-test-run-nixbot-gitlab> gitlab # [ 223.780310] nixbot[3228]: INFO: ::ffff:127.0.0.1:56960 - "GET /api/repos/gitlab/root/test-flake/builds?commit=f236896cb5377ee51df845bb2a34d4215b03d088 HTTP/1.1" 200 OK vm-test-run-nixbot-gitlab> gitlab # [ 223.781574] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:07 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 55 vm-test-run-nixbot-gitlab> gitlab # [ 223.782548] gitlab-workhorse[1288]: time="2026-06-16T09:17:07Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 vm-test-run-nixbot-gitlab> gitlab # [ 223.784242] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:07 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 70 vm-test-run-nixbot-gitlab> gitlab # [ 223.784913] gitlab-workhorse[1288]: time="2026-06-16T09:17:07Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=383 vm-test-run-nixbot-gitlab> gitlab # [ 223.786266] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:07 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 383 "" "git/2.54.0" 69 vm-test-run-nixbot-gitlab> gitlab # [ 223.787334] sidekiq-cluster[2167]: {"severity":"WARN","time":"2026-06-16T09:17:07.021Z","class":"CreatePipelineWorker","correlation_id":"01KV7VGKRGAKHYVHRQQQ4T7P98","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf","after":"f236896cb5377ee51df845bb2a34d4215b03d088","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"f236896cb5377ee51df845bb2a34d4215b03d088","ref":"refs/heads/master"},"retry":0} vm-test-run-nixbot-gitlab> gitlab # [ 223.788170] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.026Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"34622e6046e92a3090b20d8b","created_at":"2026-06-16T09:17:04.788Z","trace_propagation_headers":{"sentry-trace":"2d19cdc3dd144000aa4c9940b7686f4d-898a4ed89fe94159","baggage":"sentry-trace_id=2d19cdc3dd144000aa4c9940b7686f4d,sentry-release=Unknown"},"correlation_id":"01KV7VGKRGAKHYVHRQQQ4T7P98","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:935f70ed74e4002b458115b6d00f4313d33c954f54732e02826c2b70f53f80e5","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.790Z","job_size_bytes":467,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-34622e6046e92a3090b20d8b: done: 0.664101 sec","job_status":"done","queue_duration_s":1.571325,"scheduling_latency_s":1.571349,"gitaly_calls":7,"gitaly_duration_s":0.059282,"redis_calls":33,"redis_duration_s":0.006528,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.00096,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001243,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003025,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.0013,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":30,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.15,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":26,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.148,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.444795,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.664101,"completed_at":"2026-06-16T09:17:07.026Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.150139,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 223.862293] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.028Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"4b47565f-68ab-46a2-8de4-dcf3340c5835\"}"],"class":"WebHookWorker","jid":"343ea3d0eaf0bc04d40f7aef","created_at":"2026-06-16T09:17:04.874Z","trace_propagation_headers":{"sentry-trace":"2d19cdc3dd144000aa4c9940b7686f4d-898a4ed89fe94159","baggage":"sentry-trace_id=2d19cdc3dd144000aa4c9940b7686f4d,sentry-release=Unknown"},"correlation_id":"01KV7VGKRGAKHYVHRQQQ4T7P98","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:05bcec610550427ac9c683066731c5909b33aa4fff679f859a9aa93bec63d3b6","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.880Z","job_size_bytes":1749,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-343ea3d0eaf0bc04d40f7aef: start","job_status":"start","queue_duration_s":2.147627,"scheduling_latency_s":2.147656} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=f236896cb5377ee51df845bb2a34d4215b03d088', in 0.35 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-06-16T09:17:07.356207Z', 'started_at': None, 'finished_at': None}] vm-test-run-nixbot-gitlab> (finished: subtest: push delivers the webhook, in 4.91 seconds) vm-test-run-nixbot-gitlab> subtest: eval and build post commit statuses vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses vm-test-run-nixbot-gitlab> gitlab # [ 223.868978] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.121Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"4b47565f-68ab-46a2-8de4-dcf3340c5835\"}"],"class":"WebHookWorker","jid":"343ea3d0eaf0bc04d40f7aef","created_at":"2026-06-16T09:17:04.874Z","trace_propagation_headers":{"sentry-trace":"2d19cdc3dd144000aa4c9940b7686f4d-898a4ed89fe94159","baggage":"sentry-trace_id=2d19cdc3dd144000aa4c9940b7686f4d,sentry-release=Unknown"},"correlation_id":"01KV7VGKRGAKHYVHRQQQ4T7P98","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:05bcec610550427ac9c683066731c5909b33aa4fff679f859a9aa93bec63d3b6","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:04.880Z","job_size_bytes":1749,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-343ea3d0eaf0bc04d40f7aef: done: 0.093328 sec","job_status":"done","queue_duration_s":2.147627,"scheduling_latency_s":2.147656,"redis_calls":19,"redis_duration_s":0.005248,"redis_read_bytes":32,"redis_write_bytes":5282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00164,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000327,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3644,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00238,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.008377321000011761,"cpu_s":0.07128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000385803,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.093328,"completed_at":"2026-06-16T09:17:07.121Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002038,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses, in 0.09 seconds) vm-test-run-nixbot-gitlab> [] vm-test-run-nixbot-gitlab> gitlab # [ 223.958446] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"036beb27773497bc063a5cb5","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-036beb27773497bc063a5cb5: start","job_status":"start","queue_duration_s":2.101556,"scheduling_latency_s":2.101581} vm-test-run-nixbot-gitlab> gitlab # [ 223.966734] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"036beb27773497bc063a5cb5","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-036beb27773497bc063a5cb5: done: 0.023087 sec","job_status":"done","queue_duration_s":2.101556,"scheduling_latency_s":2.101581,"redis_calls":15,"redis_duration_s":0.003747,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000448,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001073,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002226,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023087,"completed_at":"2026-06-16T09:17:07.146Z","load_balancing_strategy":"primary","db_duration_s":0.001363,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.090303] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7df17ac7e236bf93ee602c5","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7df17ac7e236bf93ee602c5: start","job_status":"start","queue_duration_s":2.126027,"scheduling_latency_s":2.12605} vm-test-run-nixbot-gitlab> gitlab # [ 224.091255] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7df17ac7e236bf93ee602c5","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7df17ac7e236bf93ee602c5: done: 0.016129 sec","job_status":"done","queue_duration_s":2.126027,"scheduling_latency_s":2.12605,"redis_calls":13,"redis_duration_s":0.003099,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002154,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009815,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016129,"completed_at":"2026-06-16T09:17:07.163Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.170317] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc3cad1ce0bb95693b81ad06","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc3cad1ce0bb95693b81ad06: start","job_status":"start","queue_duration_s":2.143741,"scheduling_latency_s":2.143763} vm-test-run-nixbot-gitlab> gitlab # [ 224.172127] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc3cad1ce0bb95693b81ad06","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc3cad1ce0bb95693b81ad06: done: 0.024339 sec","job_status":"done","queue_duration_s":2.143741,"scheduling_latency_s":2.143763,"redis_calls":13,"redis_duration_s":0.006034,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003527,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024339,"completed_at":"2026-06-16T09:17:07.189Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.197320] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"085ec80300e31dec5ebc0ee8","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-085ec80300e31dec5ebc0ee8: start","job_status":"start","queue_duration_s":2.16975,"scheduling_latency_s":2.169849} vm-test-run-nixbot-gitlab> gitlab # [ 224.275078] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"085ec80300e31dec5ebc0ee8","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-085ec80300e31dec5ebc0ee8: done: 0.016826 sec","job_status":"done","queue_duration_s":2.16975,"scheduling_latency_s":2.169849,"redis_calls":13,"redis_duration_s":0.003372,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001033,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002339,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016826,"completed_at":"2026-06-16T09:17:07.208Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.348094] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20ac77e6e5c517ec0b89559b","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20ac77e6e5c517ec0b89559b: start","job_status":"start","queue_duration_s":2.18791,"scheduling_latency_s":2.187933} vm-test-run-nixbot-gitlab> gitlab # [ 224.349408] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20ac77e6e5c517ec0b89559b","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20ac77e6e5c517ec0b89559b: done: 0.015285 sec","job_status":"done","queue_duration_s":2.18791,"scheduling_latency_s":2.187933,"redis_calls":13,"redis_duration_s":0.003065,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000995,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015285,"completed_at":"2026-06-16T09:17:07.224Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.427914] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c92a48db8c4fb3a2a9dcd299","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c92a48db8c4fb3a2a9dcd299: start","job_status":"start","queue_duration_s":2.204949,"scheduling_latency_s":2.204972} vm-test-run-nixbot-gitlab> gitlab # [ 224.429073] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.245Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c92a48db8c4fb3a2a9dcd299","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c92a48db8c4fb3a2a9dcd299: done: 0.018655 sec","job_status":"done","queue_duration_s":2.204949,"scheduling_latency_s":2.204972,"redis_calls":13,"redis_duration_s":0.005834,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001031,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004803,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018655,"completed_at":"2026-06-16T09:17:07.245Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.504149] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d002c9aa615395d813c9abf","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d002c9aa615395d813c9abf: start","job_status":"start","queue_duration_s":2.224903,"scheduling_latency_s":2.224925} vm-test-run-nixbot-gitlab> gitlab # [ 224.505743] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d002c9aa615395d813c9abf","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d002c9aa615395d813c9abf: done: 0.0176 sec","job_status":"done","queue_duration_s":2.224903,"scheduling_latency_s":2.224925,"redis_calls":13,"redis_duration_s":0.003939,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001373,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002566,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0176,"completed_at":"2026-06-16T09:17:07.264Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.578289] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08f1c733e04cb5b8a5b0cc95","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08f1c733e04cb5b8a5b0cc95: start","job_status":"start","queue_duration_s":2.243908,"scheduling_latency_s":2.243931} vm-test-run-nixbot-gitlab> gitlab # [ 224.579363] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08f1c733e04cb5b8a5b0cc95","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08f1c733e04cb5b8a5b0cc95: done: 0.015853 sec","job_status":"done","queue_duration_s":2.243908,"scheduling_latency_s":2.243931,"redis_calls":13,"redis_duration_s":0.003279,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002224,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015853,"completed_at":"2026-06-16T09:17:07.281Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.657350] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cddf75456a0314c2f4168a73","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cddf75456a0314c2f4168a73: start","job_status":"start","queue_duration_s":2.26105,"scheduling_latency_s":2.261079} vm-test-run-nixbot-gitlab> gitlab # [ 224.659421] gitaly[1373]: time="2026-06-16T09:17:06.258Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGMF5BNRJZGXMK1Q8F294 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:36.157" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:06.257" grpc.time_ms=0.817 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 224.660372] gitaly[1373]: time="2026-06-16T09:17:06.342Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 diskcache=6f293df5-57bb-4aa9-a886-f3735eff61ca grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-16T15:17:06.310" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:06.311" pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 224.729254] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses HTTP/1.1" 200 2 "" "curl/8.20.0" 62 vm-test-run-nixbot-gitlab> gitlab # [ 224.730728] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f236896cb5377ee51df845bb2a34d4215b03d088 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 1221 vm-test-run-nixbot-gitlab> gitlab # [ 224.731963] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cddf75456a0314c2f4168a73","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cddf75456a0314c2f4168a73: done: 0.018 sec","job_status":"done","queue_duration_s":2.26105,"scheduling_latency_s":2.261079,"redis_calls":13,"redis_duration_s":0.005513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001162,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004351,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018,"completed_at":"2026-06-16T09:17:07.300Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.734379] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.306Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b70d501ad3760d9b406ecbc5","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b70d501ad3760d9b406ecbc5: start","job_status":"start","queue_duration_s":2.284534,"scheduling_latency_s":2.284561} vm-test-run-nixbot-gitlab> gitlab # [ 224.807144] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.329Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b70d501ad3760d9b406ecbc5","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b70d501ad3760d9b406ecbc5: done: 0.023502 sec","job_status":"done","queue_duration_s":2.284534,"scheduling_latency_s":2.284561,"redis_calls":13,"redis_duration_s":0.007912,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023502,"completed_at":"2026-06-16T09:17:07.329Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.816360] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"349abe9e459f10bf014d6e6f","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-349abe9e459f10bf014d6e6f: start","job_status":"start","queue_duration_s":2.310104,"scheduling_latency_s":2.310127} vm-test-run-nixbot-gitlab> gitlab # [ 224.899460] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"349abe9e459f10bf014d6e6f","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-349abe9e459f10bf014d6e6f: done: 0.023953 sec","job_status":"done","queue_duration_s":2.310104,"scheduling_latency_s":2.310127,"redis_calls":13,"redis_duration_s":0.004624,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003469,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023953,"completed_at":"2026-06-16T09:17:07.355Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.901863] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e5b81381bdeaa961f1eb36e","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e5b81381bdeaa961f1eb36e: start","job_status":"start","queue_duration_s":2.336939,"scheduling_latency_s":2.336969} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses vm-test-run-nixbot-gitlab> gitlab # [ 224.917943] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.394Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e5b81381bdeaa961f1eb36e","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e5b81381bdeaa961f1eb36e: done: 0.035719 sec","job_status":"done","queue_duration_s":2.336939,"scheduling_latency_s":2.336969,"redis_calls":13,"redis_duration_s":0.017401,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001021,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.01638,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035719,"completed_at":"2026-06-16T09:17:07.394Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 224.982406] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fea355c72f54ab169ba61eb8","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fea355c72f54ab169ba61eb8: start","job_status":"start","queue_duration_s":2.375492,"scheduling_latency_s":2.375526} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses, in 0.09 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-16T09:17:08.562Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 225.002256] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fea355c72f54ab169ba61eb8","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fea355c72f54ab169ba61eb8: done: 0.016327 sec","job_status":"done","queue_duration_s":2.375492,"scheduling_latency_s":2.375526,"redis_calls":13,"redis_duration_s":0.003064,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000944,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00212,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016327,"completed_at":"2026-06-16T09:17:07.413Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.082072] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a31792a71860910dbff9a2d2","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a31792a71860910dbff9a2d2: start","job_status":"start","queue_duration_s":2.393306,"scheduling_latency_s":2.393328} vm-test-run-nixbot-gitlab> gitlab # [ 225.083378] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a31792a71860910dbff9a2d2","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a31792a71860910dbff9a2d2: done: 0.015432 sec","job_status":"done","queue_duration_s":2.393306,"scheduling_latency_s":2.393328,"redis_calls":13,"redis_duration_s":0.0030759999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015432,"completed_at":"2026-06-16T09:17:07.430Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.161214] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a5952e612f5241162a22eb5","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a5952e612f5241162a22eb5: start","job_status":"start","queue_duration_s":2.410127,"scheduling_latency_s":2.41015} vm-test-run-nixbot-gitlab> gitlab # [ 225.164206] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.447Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a5952e612f5241162a22eb5","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a5952e612f5241162a22eb5: done: 0.015454 sec","job_status":"done","queue_duration_s":2.410127,"scheduling_latency_s":2.41015,"redis_calls":13,"redis_duration_s":0.0030220000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015454,"completed_at":"2026-06-16T09:17:07.447Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.264590] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"183ad15c6a25c0acf102a8e2","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-183ad15c6a25c0acf102a8e2: start","job_status":"start","queue_duration_s":2.42696,"scheduling_latency_s":2.426982} vm-test-run-nixbot-gitlab> gitlab # [ 225.266101] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"183ad15c6a25c0acf102a8e2","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-183ad15c6a25c0acf102a8e2: done: 0.015653 sec","job_status":"done","queue_duration_s":2.42696,"scheduling_latency_s":2.426982,"redis_calls":13,"redis_duration_s":0.003065,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009735,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015653,"completed_at":"2026-06-16T09:17:07.464Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.362353] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"955572d3b08b88b0960c1970","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-955572d3b08b88b0960c1970: start","job_status":"start","queue_duration_s":2.444046,"scheduling_latency_s":2.444068} vm-test-run-nixbot-gitlab> gitlab # [ 225.363917] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.482Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"955572d3b08b88b0960c1970","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-955572d3b08b88b0960c1970: done: 0.016593 sec","job_status":"done","queue_duration_s":2.444046,"scheduling_latency_s":2.444068,"redis_calls":13,"redis_duration_s":0.003059,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001067,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010183,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016593,"completed_at":"2026-06-16T09:17:07.482Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.443319] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3085ea12a4f2711c0a6bcc22","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3085ea12a4f2711c0a6bcc22: start","job_status":"start","queue_duration_s":2.462198,"scheduling_latency_s":2.462221} vm-test-run-nixbot-gitlab> gitlab # [ 225.444876] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.512Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3085ea12a4f2711c0a6bcc22","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3085ea12a4f2711c0a6bcc22: done: 0.028482 sec","job_status":"done","queue_duration_s":2.462198,"scheduling_latency_s":2.462221,"redis_calls":13,"redis_duration_s":0.012209,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.010657,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011672,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028482,"completed_at":"2026-06-16T09:17:07.512Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.530250] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16aae0c0c616175da4de01c0","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16aae0c0c616175da4de01c0: start","job_status":"start","queue_duration_s":2.492173,"scheduling_latency_s":2.492197} vm-test-run-nixbot-gitlab> gitlab # [ 225.531499] nixbot[3228]: {"ts": "2026-06-16T09:17:08.591Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f236896cb5377ee51df845bb2a34d4215b03d088 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 225.532465] gitaly[1373]: time="2026-06-16T09:17:06.342Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1647 command.oublock=0 command.real_time_ms=32 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-06-16T15:17:06.310" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:06.311" grpc.time_ms=31.308 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 225.533363] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.530Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16aae0c0c616175da4de01c0","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16aae0c0c616175da4de01c0: done: 0.016089 sec","job_status":"done","queue_duration_s":2.492173,"scheduling_latency_s":2.492197,"redis_calls":13,"redis_duration_s":0.0029389999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002005,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016089,"completed_at":"2026-06-16T09:17:07.529Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.615747] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.531Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c99651d4efa624c0ee10f382","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c99651d4efa624c0ee10f382: start","job_status":"start","queue_duration_s":2.509654,"scheduling_latency_s":2.509676} vm-test-run-nixbot-gitlab> gitlab # [ 225.617095] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.547Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c99651d4efa624c0ee10f382","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c99651d4efa624c0ee10f382: done: 0.01597 sec","job_status":"done","queue_duration_s":2.509654,"scheduling_latency_s":2.509676,"redis_calls":13,"redis_duration_s":0.003261,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00108,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002181,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01597,"completed_at":"2026-06-16T09:17:07.547Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.686208] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95ee8fc659431381310c5d13","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95ee8fc659431381310c5d13: start","job_status":"start","queue_duration_s":2.526992,"scheduling_latency_s":2.527013} vm-test-run-nixbot-gitlab> gitlab # [ 225.687730] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95ee8fc659431381310c5d13","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95ee8fc659431381310c5d13: done: 0.020229 sec","job_status":"done","queue_duration_s":2.526992,"scheduling_latency_s":2.527013,"redis_calls":13,"redis_duration_s":0.005013999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00208,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020229,"completed_at":"2026-06-16T09:17:07.568Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.743288] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d70352d3bb2d35acfc1b3eeb","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d70352d3bb2d35acfc1b3eeb: start","job_status":"start","queue_duration_s":2.550816,"scheduling_latency_s":2.550847} vm-test-run-nixbot-gitlab> gitlab # [ 225.763257] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d70352d3bb2d35acfc1b3eeb","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d70352d3bb2d35acfc1b3eeb: done: 0.021515 sec","job_status":"done","queue_duration_s":2.550816,"scheduling_latency_s":2.550847,"redis_calls":13,"redis_duration_s":0.004844,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00236,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002484,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013452,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021515,"completed_at":"2026-06-16T09:17:07.594Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.866075] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68eadaa5ed9db34675d9c2f5","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68eadaa5ed9db34675d9c2f5: start","job_status":"start","queue_duration_s":2.573946,"scheduling_latency_s":2.57398} vm-test-run-nixbot-gitlab> gitlab # [ 225.867333] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68eadaa5ed9db34675d9c2f5","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68eadaa5ed9db34675d9c2f5: done: 0.016379 sec","job_status":"done","queue_duration_s":2.573946,"scheduling_latency_s":2.57398,"redis_calls":13,"redis_duration_s":0.002886,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016379,"completed_at":"2026-06-16T09:17:07.612Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.907452] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e603b68c19d486457d2ff9f","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e603b68c19d486457d2ff9f: start","job_status":"start","queue_duration_s":2.591758,"scheduling_latency_s":2.59178} vm-test-run-nixbot-gitlab> gitlab # [ 225.977409] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.628Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e603b68c19d486457d2ff9f","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e603b68c19d486457d2ff9f: done: 0.015351 sec","job_status":"done","queue_duration_s":2.591758,"scheduling_latency_s":2.59178,"redis_calls":13,"redis_duration_s":0.00281,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015351,"completed_at":"2026-06-16T09:17:07.628Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 225.990827] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28d8c772232a6555054d8d1f","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28d8c772232a6555054d8d1f: start","job_status":"start","queue_duration_s":2.608457,"scheduling_latency_s":2.60848} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses vm-test-run-nixbot-gitlab> gitlab # [ 226.077131] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.645Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28d8c772232a6555054d8d1f","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28d8c772232a6555054d8d1f: done: 0.015672 sec","job_status":"done","queue_duration_s":2.608457,"scheduling_latency_s":2.60848,"redis_calls":13,"redis_duration_s":0.003164,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0011,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015672,"completed_at":"2026-06-16T09:17:07.645Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.079070] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d40ba4891dab16898d3bfe90","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d40ba4891dab16898d3bfe90: start","job_status":"start","queue_duration_s":2.625536,"scheduling_latency_s":2.62556} vm-test-run-nixbot-gitlab> gitlab # [ 226.091399] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.665Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d40ba4891dab16898d3bfe90","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d40ba4891dab16898d3bfe90: done: 0.017734 sec","job_status":"done","queue_duration_s":2.625536,"scheduling_latency_s":2.62556,"redis_calls":13,"redis_duration_s":0.005212,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003668,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010252,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017734,"completed_at":"2026-06-16T09:17:07.665Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses, in 0.10 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-16T09:17:08.562Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 226.159129] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba66f069a9232b3d1164083e","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba66f069a9232b3d1164083e: start","job_status":"start","queue_duration_s":2.645638,"scheduling_latency_s":2.645661} vm-test-run-nixbot-gitlab> gitlab # [ 226.230899] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba66f069a9232b3d1164083e","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba66f069a9232b3d1164083e: done: 0.020725 sec","job_status":"done","queue_duration_s":2.645638,"scheduling_latency_s":2.645661,"redis_calls":13,"redis_duration_s":0.007525,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020725,"completed_at":"2026-06-16T09:17:07.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.233617] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bdaf907628cf26c01c09dcf","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bdaf907628cf26c01c09dcf: start","job_status":"start","queue_duration_s":2.669621,"scheduling_latency_s":2.669732} vm-test-run-nixbot-gitlab> gitlab # [ 226.303779] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.718Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bdaf907628cf26c01c09dcf","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bdaf907628cf26c01c09dcf: done: 0.027353 sec","job_status":"done","queue_duration_s":2.669621,"scheduling_latency_s":2.669732,"redis_calls":13,"redis_duration_s":0.003793,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002702,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027353,"completed_at":"2026-06-16T09:17:07.718Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.314129] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"357582620c9962984bae9730","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-357582620c9962984bae9730: start","job_status":"start","queue_duration_s":2.698536,"scheduling_latency_s":2.698565} vm-test-run-nixbot-gitlab> gitlab # [ 226.400703] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:08 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 51 vm-test-run-nixbot-gitlab> gitlab # [ 226.401972] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:10 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 68 vm-test-run-nixbot-gitlab> gitlab # [ 226.403499] gitaly[1373]: time="2026-06-16T09:17:06.384Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:36.283" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:06.383" grpc.time_ms=0.801 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 226.404432] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"357582620c9962984bae9730","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-357582620c9962984bae9730: done: 0.020126 sec","job_status":"done","queue_duration_s":2.698536,"scheduling_latency_s":2.698565,"redis_calls":13,"redis_duration_s":0.0050290000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001033,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020126,"completed_at":"2026-06-16T09:17:07.740Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.473113] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95f3bbe4d9343096f2d2ad5a","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95f3bbe4d9343096f2d2ad5a: start","job_status":"start","queue_duration_s":2.720287,"scheduling_latency_s":2.720318} vm-test-run-nixbot-gitlab> gitlab # [ 226.474175] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95f3bbe4d9343096f2d2ad5a","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95f3bbe4d9343096f2d2ad5a: done: 0.016805 sec","job_status":"done","queue_duration_s":2.720287,"scheduling_latency_s":2.720318,"redis_calls":13,"redis_duration_s":0.0031190000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002196,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016805,"completed_at":"2026-06-16T09:17:07.758Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.505137] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddd826619a85256e50c4a7fc","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddd826619a85256e50c4a7fc: start","job_status":"start","queue_duration_s":2.738462,"scheduling_latency_s":2.738486} vm-test-run-nixbot-gitlab> gitlab # [ 226.525182] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ddd826619a85256e50c4a7fc","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ddd826619a85256e50c4a7fc: done: 0.018341 sec","job_status":"done","queue_duration_s":2.738462,"scheduling_latency_s":2.738486,"redis_calls":13,"redis_duration_s":0.003273,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001024,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002249,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018341,"completed_at":"2026-06-16T09:17:07.778Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.605078] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.780Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"653bd0225c4370f767e40bb4","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-653bd0225c4370f767e40bb4: start","job_status":"start","queue_duration_s":2.758288,"scheduling_latency_s":2.75831} vm-test-run-nixbot-gitlab> gitlab # [ 226.606130] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"653bd0225c4370f767e40bb4","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-653bd0225c4370f767e40bb4: done: 0.020905 sec","job_status":"done","queue_duration_s":2.758288,"scheduling_latency_s":2.75831,"redis_calls":13,"redis_duration_s":0.004071,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002573,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020905,"completed_at":"2026-06-16T09:17:07.800Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.669149] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04bc54fb6b08ad531f712459","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04bc54fb6b08ad531f712459: start","job_status":"start","queue_duration_s":2.780858,"scheduling_latency_s":2.780902} vm-test-run-nixbot-gitlab> gitlab # [ 226.752884] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04bc54fb6b08ad531f712459","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04bc54fb6b08ad531f712459: done: 0.018885 sec","job_status":"done","queue_duration_s":2.780858,"scheduling_latency_s":2.780902,"redis_calls":13,"redis_duration_s":0.003275,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018885,"completed_at":"2026-06-16T09:17:07.821Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.754992] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.822Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac072196087e9b65dc97f617","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac072196087e9b65dc97f617: start","job_status":"start","queue_duration_s":2.801136,"scheduling_latency_s":2.801158} vm-test-run-nixbot-gitlab> gitlab # [ 226.769891] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.841Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac072196087e9b65dc97f617","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac072196087e9b65dc97f617: done: 0.018225 sec","job_status":"done","queue_duration_s":2.801136,"scheduling_latency_s":2.801158,"redis_calls":13,"redis_duration_s":0.003854,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00108,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002774,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018225,"completed_at":"2026-06-16T09:17:07.841Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.893305] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52c883a4fd1b5b1a3dca234b","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52c883a4fd1b5b1a3dca234b: start","job_status":"start","queue_duration_s":2.820896,"scheduling_latency_s":2.820923} vm-test-run-nixbot-gitlab> gitlab # [ 226.894847] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52c883a4fd1b5b1a3dca234b","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52c883a4fd1b5b1a3dca234b: done: 0.022404 sec","job_status":"done","queue_duration_s":2.820896,"scheduling_latency_s":2.820923,"redis_calls":13,"redis_duration_s":0.0032969999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001034,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002263,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022404,"completed_at":"2026-06-16T09:17:07.865Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 226.914070] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bc1adca39a4241ebd315992","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bc1adca39a4241ebd315992: start","job_status":"start","queue_duration_s":2.844865,"scheduling_latency_s":2.844889} vm-test-run-nixbot-gitlab> gitlab # [ 226.934137] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bc1adca39a4241ebd315992","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bc1adca39a4241ebd315992: done: 0.113479 sec","job_status":"done","queue_duration_s":2.844865,"scheduling_latency_s":2.844889,"redis_calls":13,"redis_duration_s":0.003764,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001382,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002382,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.113479,"completed_at":"2026-06-16T09:17:07.980Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.011076] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:07.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdf32c6b67c34834b0e32d1e","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdf32c6b67c34834b0e32d1e: start","job_status":"start","queue_duration_s":2.961088,"scheduling_latency_s":2.961119} vm-test-run-nixbot-gitlab> gitlab # [ 227.015217] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.001Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdf32c6b67c34834b0e32d1e","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdf32c6b67c34834b0e32d1e: done: 0.018688 sec","job_status":"done","queue_duration_s":2.961088,"scheduling_latency_s":2.961119,"redis_calls":13,"redis_duration_s":0.0029909999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000995,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018688,"completed_at":"2026-06-16T09:17:08.001Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.079122] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84ce1cee67ff1ca547832952","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84ce1cee67ff1ca547832952: start","job_status":"start","queue_duration_s":2.981,"scheduling_latency_s":2.981021} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses vm-test-run-nixbot-gitlab> gitlab # [ 227.099121] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.018Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84ce1cee67ff1ca547832952","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84ce1cee67ff1ca547832952: done: 0.015442 sec","job_status":"done","queue_duration_s":2.981,"scheduling_latency_s":2.981021,"redis_calls":13,"redis_duration_s":0.002765,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001881,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010431,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015442,"completed_at":"2026-06-16T09:17:08.018Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.168074] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8786d133edd615f6e7f0af6a","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8786d133edd615f6e7f0af6a: start","job_status":"start","queue_duration_s":2.997692,"scheduling_latency_s":2.997714} vm-test-run-nixbot-gitlab> gitlab # [ 227.180086] gitaly[1373]: time="2026-06-16T09:17:06.393Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-16T09:17:16.006" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:06.386" grpc.time_ms=6.287 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses, in 0.10 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-16T09:17:08.562Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 227.253065] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8786d133edd615f6e7f0af6a","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8786d133edd615f6e7f0af6a: done: 0.014778 sec","job_status":"done","queue_duration_s":2.997692,"scheduling_latency_s":2.997714,"redis_calls":13,"redis_duration_s":0.002634,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000829,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001805,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014778,"completed_at":"2026-06-16T09:17:08.034Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.261247] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.113Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c61f85bdb7b7dd1a5ede7e59","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c61f85bdb7b7dd1a5ede7e59: start","job_status":"start","queue_duration_s":3.091872,"scheduling_latency_s":3.091909} vm-test-run-nixbot-gitlab> gitlab # [ 227.334377] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c61f85bdb7b7dd1a5ede7e59","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c61f85bdb7b7dd1a5ede7e59: done: 0.017651 sec","job_status":"done","queue_duration_s":3.091872,"scheduling_latency_s":3.091909,"redis_calls":13,"redis_duration_s":0.002759,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017651,"completed_at":"2026-06-16T09:17:08.131Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.343923] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de3394ae0a5e64dceef99594","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de3394ae0a5e64dceef99594: start","job_status":"start","queue_duration_s":3.110673,"scheduling_latency_s":3.110695} vm-test-run-nixbot-gitlab> gitlab # [ 227.363959] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de3394ae0a5e64dceef99594","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de3394ae0a5e64dceef99594: done: 0.015832 sec","job_status":"done","queue_duration_s":3.110673,"scheduling_latency_s":3.110695,"redis_calls":13,"redis_duration_s":0.003251,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002451,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015832,"completed_at":"2026-06-16T09:17:08.148Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.427341] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3afb47a76d6acfd6c5618d2a","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3afb47a76d6acfd6c5618d2a: start","job_status":"start","queue_duration_s":3.12772,"scheduling_latency_s":3.127742} vm-test-run-nixbot-gitlab> gitlab # [ 227.510136] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.164Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3afb47a76d6acfd6c5618d2a","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3afb47a76d6acfd6c5618d2a: done: 0.015253 sec","job_status":"done","queue_duration_s":3.12772,"scheduling_latency_s":3.127742,"redis_calls":13,"redis_duration_s":0.00294,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010214,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015253,"completed_at":"2026-06-16T09:17:08.164Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.512004] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ea039e78943a8e335aae419","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ea039e78943a8e335aae419: start","job_status":"start","queue_duration_s":3.144186,"scheduling_latency_s":3.14465} vm-test-run-nixbot-gitlab> gitlab # [ 227.528095] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ea039e78943a8e335aae419","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ea039e78943a8e335aae419: done: 0.015819 sec","job_status":"done","queue_duration_s":3.144186,"scheduling_latency_s":3.14465,"redis_calls":13,"redis_duration_s":0.003472,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001428,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015819,"completed_at":"2026-06-16T09:17:08.181Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.591389] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08cb9987b85f601c49b78a83","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08cb9987b85f601c49b78a83: start","job_status":"start","queue_duration_s":3.161335,"scheduling_latency_s":3.161405} vm-test-run-nixbot-gitlab> gitlab # [ 227.611234] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08cb9987b85f601c49b78a83","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08cb9987b85f601c49b78a83: done: 0.015418 sec","job_status":"done","queue_duration_s":3.161335,"scheduling_latency_s":3.161405,"redis_calls":13,"redis_duration_s":0.0031090000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002249,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015418,"completed_at":"2026-06-16T09:17:08.198Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.674472] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52666dace8e01cf96bd89169","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52666dace8e01cf96bd89169: start","job_status":"start","queue_duration_s":3.178037,"scheduling_latency_s":3.178058} vm-test-run-nixbot-gitlab> gitlab # [ 227.742294] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52666dace8e01cf96bd89169","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52666dace8e01cf96bd89169: done: 0.015272 sec","job_status":"done","queue_duration_s":3.178037,"scheduling_latency_s":3.178058,"redis_calls":13,"redis_duration_s":0.003075,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001035,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00204,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015272,"completed_at":"2026-06-16T09:17:08.215Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.755165] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fbe190b128942399e39298a","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fbe190b128942399e39298a: start","job_status":"start","queue_duration_s":3.19509,"scheduling_latency_s":3.195112} vm-test-run-nixbot-gitlab> gitlab # [ 227.823174] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fbe190b128942399e39298a","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fbe190b128942399e39298a: done: 0.018612 sec","job_status":"done","queue_duration_s":3.19509,"scheduling_latency_s":3.195112,"redis_calls":13,"redis_duration_s":0.003118,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00112,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018612,"completed_at":"2026-06-16T09:17:08.235Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.836077] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a2758e2db4c3784cab1d761","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a2758e2db4c3784cab1d761: start","job_status":"start","queue_duration_s":3.215975,"scheduling_latency_s":3.216012} vm-test-run-nixbot-gitlab> gitlab # [ 227.910146] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a2758e2db4c3784cab1d761","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a2758e2db4c3784cab1d761: done: 0.017933 sec","job_status":"done","queue_duration_s":3.215975,"scheduling_latency_s":3.216012,"redis_calls":13,"redis_duration_s":0.003072,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000967,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017933,"completed_at":"2026-06-16T09:17:08.255Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.916937] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"950f9ab266baba7dd627ca42","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-950f9ab266baba7dd627ca42: start","job_status":"start","queue_duration_s":3.235215,"scheduling_latency_s":3.235237} vm-test-run-nixbot-gitlab> gitlab # [ 227.992218] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"950f9ab266baba7dd627ca42","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-950f9ab266baba7dd627ca42: done: 0.015912 sec","job_status":"done","queue_duration_s":3.235215,"scheduling_latency_s":3.235237,"redis_calls":13,"redis_duration_s":0.002889,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015912,"completed_at":"2026-06-16T09:17:08.272Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 227.997800] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe0f3f33b359d4dcef15f16d","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe0f3f33b359d4dcef15f16d: start","job_status":"start","queue_duration_s":3.25246,"scheduling_latency_s":3.252879} vm-test-run-nixbot-gitlab> gitlab # [ 228.066583] gitaly[1373]: time="2026-06-16T09:17:06.420Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-16T09:17:36.318" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:06.419" grpc.time_ms=0.94 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 228.068115] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 62 vm-test-run-nixbot-gitlab> gitlab # [ 228.068936] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe0f3f33b359d4dcef15f16d","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe0f3f33b359d4dcef15f16d: done: 0.015993 sec","job_status":"done","queue_duration_s":3.25246,"scheduling_latency_s":3.252879,"redis_calls":13,"redis_duration_s":0.0029100000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010516,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015993,"completed_at":"2026-06-16T09:17:08.290Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.166254] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.291Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c1e011e0040706c6a01c0ee","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c1e011e0040706c6a01c0ee: start","job_status":"start","queue_duration_s":3.269754,"scheduling_latency_s":3.269775} vm-test-run-nixbot-gitlab> gitlab # [ 228.167298] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c1e011e0040706c6a01c0ee","created_at":"2026-06-16T09:17:04.958Z","trace_propagation_headers":{"sentry-trace":"92e0bc9f7e9e4a16b0ef0327a955618a-b8d49923c91d48cf","baggage":"sentry-trace_id=92e0bc9f7e9e4a16b0ef0327a955618a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1d408c99ae6f7e8b3bd16df7a18f891b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-06-16T09:17:05.021Z","job_size_bytes":2,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c1e011e0040706c6a01c0ee: done: 0.015486 sec","job_status":"done","queue_duration_s":3.269754,"scheduling_latency_s":3.269775,"redis_calls":13,"redis_duration_s":0.002941,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00097,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010437,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015486,"completed_at":"2026-06-16T09:17:08.306Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.184892] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.308Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"9f3c119c79a9f3173d0d80f4","created_at":"2026-06-16T09:17:06.177Z","trace_propagation_headers":{"sentry-trace":"37688b26f51c473b96d8e2d67a015395-98a9e4239e364c13","baggage":"sentry-trace_id=37688b26f51c473b96d8e2d67a015395,sentry-release=Unknown"},"correlation_id":"01KV7VGMF5BNRJZGXMK1Q8F294","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:06.182Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9f3c119c79a9f3173d0d80f4: start","job_status":"start","queue_duration_s":2.125668,"scheduling_latency_s":2.125692} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses vm-test-run-nixbot-gitlab> gitlab # [ 228.210824] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.343Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"9f3c119c79a9f3173d0d80f4","created_at":"2026-06-16T09:17:06.177Z","trace_propagation_headers":{"sentry-trace":"37688b26f51c473b96d8e2d67a015395-98a9e4239e364c13","baggage":"sentry-trace_id=37688b26f51c473b96d8e2d67a015395,sentry-release=Unknown"},"correlation_id":"01KV7VGMF5BNRJZGXMK1Q8F294","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:06.182Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9f3c119c79a9f3173d0d80f4: done: 0.034907 sec","job_status":"done","queue_duration_s":2.125668,"scheduling_latency_s":2.125692,"redis_calls":10,"redis_duration_s":0.002793,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000551,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001654,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000588,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000372115,"duration_s":0.034907,"completed_at":"2026-06-16T09:17:08.343Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004307,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.281981] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.344Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","343ea3d0eaf0bc04d40f7aef"],"class":"WebHooks::LogExecutionWorker","jid":"444a3f78659b5e282f174bc8","created_at":"2026-06-16T09:17:07.106Z","trace_propagation_headers":{"sentry-trace":"2d19cdc3dd144000aa4c9940b7686f4d-6f12711c73134e99","baggage":"sentry-trace_id=2d19cdc3dd144000aa4c9940b7686f4d,sentry-release=Unknown"},"correlation_id":"01KV7VGKRGAKHYVHRQQQ4T7P98","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a5f94f7949b36f39b96ebe01bf78eb4fba7d2bc2b21eb5d84f26539af5c18d8f","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:07.114Z","job_size_bytes":2507,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-444a3f78659b5e282f174bc8: start","job_status":"start","queue_duration_s":1.23071,"scheduling_latency_s":1.230734} vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses, in 0.09 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-16T09:17:08.562Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 228.363202] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.460Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","343ea3d0eaf0bc04d40f7aef"],"class":"WebHooks::LogExecutionWorker","jid":"444a3f78659b5e282f174bc8","created_at":"2026-06-16T09:17:07.106Z","trace_propagation_headers":{"sentry-trace":"2d19cdc3dd144000aa4c9940b7686f4d-6f12711c73134e99","baggage":"sentry-trace_id=2d19cdc3dd144000aa4c9940b7686f4d,sentry-release=Unknown"},"correlation_id":"01KV7VGKRGAKHYVHRQQQ4T7P98","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:a5f94f7949b36f39b96ebe01bf78eb4fba7d2bc2b21eb5d84f26539af5c18d8f","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:07.114Z","job_size_bytes":2507,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-444a3f78659b5e282f174bc8: done: 0.115933 sec","job_status":"done","queue_duration_s":1.23071,"scheduling_latency_s":1.230734,"redis_calls":10,"redis_duration_s":0.0028150000000000002,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000508,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001716,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000591,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.031,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.031,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.094758,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000465981,"duration_s":0.115933,"completed_at":"2026-06-16T09:17:08.460Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009364,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.379167] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.466Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"a32bf938c3494ad91939b81e","created_at":"2026-06-16T09:17:08.460Z","trace_propagation_headers":{"sentry-trace":"b2ee529c64fb4f088325b87d7769c55c-99599b136be94d80","baggage":"sentry-trace_id=b2ee529c64fb4f088325b87d7769c55c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGQX80H63KBAA3FAB9RX2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:08.464Z","job_size_bytes":65,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-a32bf938c3494ad91939b81e: start","job_status":"start","queue_duration_s":0.00168,"scheduling_latency_s":0.001717} vm-test-run-nixbot-gitlab> gitlab # [ 228.456150] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.493Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"a32bf938c3494ad91939b81e","created_at":"2026-06-16T09:17:08.460Z","trace_propagation_headers":{"sentry-trace":"b2ee529c64fb4f088325b87d7769c55c-99599b136be94d80","baggage":"sentry-trace_id=b2ee529c64fb4f088325b87d7769c55c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGQX80H63KBAA3FAB9RX2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:08.464Z","job_size_bytes":65,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-a32bf938c3494ad91939b81e: done: 0.027707 sec","job_status":"done","queue_duration_s":0.00168,"scheduling_latency_s":0.001717,"redis_calls":7,"redis_duration_s":0.001794,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001286,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000508,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027707,"completed_at":"2026-06-16T09:17:08.493Z","load_balancing_strategy":"primary","db_duration_s":0.004395,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 228.527322] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.495Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"91dba5ca7a1deafcec5a6214","created_at":"2026-06-16T09:17:08.466Z","trace_propagation_headers":{"sentry-trace":"b2ee529c64fb4f088325b87d7769c55c-99599b136be94d80","baggage":"sentry-trace_id=b2ee529c64fb4f088325b87d7769c55c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGQX80H63KBAA3FAB9RX2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:08.470Z","job_size_bytes":65,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-91dba5ca7a1deafcec5a6214: start","job_status":"start","queue_duration_s":0.024999,"scheduling_latency_s":0.025035} vm-test-run-nixbot-gitlab> gitlab # [ 228.527947] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.538Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"91dba5ca7a1deafcec5a6214","created_at":"2026-06-16T09:17:08.466Z","trace_propagation_headers":{"sentry-trace":"b2ee529c64fb4f088325b87d7769c55c-99599b136be94d80","baggage":"sentry-trace_id=b2ee529c64fb4f088325b87d7769c55c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGQX80H63KBAA3FAB9RX2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:08.470Z","job_size_bytes":65,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-91dba5ca7a1deafcec5a6214: done: 0.042611 sec","job_status":"done","queue_duration_s":0.024999,"scheduling_latency_s":0.025035,"redis_calls":10,"redis_duration_s":0.0025659999999999997,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001333,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001233,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042611,"completed_at":"2026-06-16T09:17:08.538Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008377,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.597776] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.574Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"bd9269a3cd3f7b8832deab10","created_at":"2026-06-16T09:17:08.569Z","trace_propagation_headers":{"sentry-trace":"b2ee529c64fb4f088325b87d7769c55c-99599b136be94d80","baggage":"sentry-trace_id=b2ee529c64fb4f088325b87d7769c55c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGQX80H63KBAA3FAB9RX2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:08.572Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-bd9269a3cd3f7b8832deab10: start","job_status":"start","queue_duration_s":0.001134,"scheduling_latency_s":0.001169} vm-test-run-nixbot-gitlab> gitlab # [ 228.599127] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.705Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"bd9269a3cd3f7b8832deab10","created_at":"2026-06-16T09:17:08.569Z","trace_propagation_headers":{"sentry-trace":"b2ee529c64fb4f088325b87d7769c55c-99599b136be94d80","baggage":"sentry-trace_id=b2ee529c64fb4f088325b87d7769c55c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGQX80H63KBAA3FAB9RX2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:08.572Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-bd9269a3cd3f7b8832deab10: done: 0.131485 sec","job_status":"done","queue_duration_s":0.001134,"scheduling_latency_s":0.001169,"gitaly_calls":3,"gitaly_duration_s":0.008355,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008272,"redis_read_bytes":98,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001057,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002349,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000297,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001739,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000272,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.00055,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000951,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":31,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":31,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":19,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.095887,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.10928456,"duration_s":0.131485,"completed_at":"2026-06-16T09:17:08.705Z","load_balancing_strategy":"primary","db_duration_s":0.025018,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 228.675076] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.707Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d95e395657cc364e139f56e9","created_at":"2026-06-16T09:17:08.625Z","trace_propagation_headers":{"sentry-trace":"b2ee529c64fb4f088325b87d7769c55c-a8453451a9c9480b","baggage":"sentry-trace_id=b2ee529c64fb4f088325b87d7769c55c,sentry-release=Unknown"},"correlation_id":"01KV7VGQX80H63KBAA3FAB9RX2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:08.628Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d95e395657cc364e139f56e9: start","job_status":"start","queue_duration_s":0.078791,"scheduling_latency_s":0.078814} vm-test-run-nixbot-gitlab> gitlab # [ 228.760166] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:08.732Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d95e395657cc364e139f56e9","created_at":"2026-06-16T09:17:08.625Z","trace_propagation_headers":{"sentry-trace":"b2ee529c64fb4f088325b87d7769c55c-a8453451a9c9480b","baggage":"sentry-trace_id=b2ee529c64fb4f088325b87d7769c55c,sentry-release=Unknown"},"correlation_id":"01KV7VGQX80H63KBAA3FAB9RX2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:08.628Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d95e395657cc364e139f56e9: done: 0.025772 sec","job_status":"done","queue_duration_s":0.078791,"scheduling_latency_s":0.078814,"redis_calls":7,"redis_duration_s":0.001751,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001232,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000519,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017569,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025772,"completed_at":"2026-06-16T09:17:08.732Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003258,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 228.767184] gitaly[1373]: time="2026-06-16T09:17:06.426Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=230 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-06-16T09:17:36.322" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:06.423" grpc.time_ms=2.963 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 228.793162] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 58 vm-test-run-nixbot-gitlab> gitlab # [ 228.797149] gitaly[1373]: time="2026-06-16T09:17:06.480Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-16T09:17:16.090" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:06.470" grpc.time_ms=9.958 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 228.865256] gitaly[1373]: time="2026-06-16T09:17:06.997Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=959 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-06-16T09:17:36.383" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:06.985" grpc.time_ms=12.141 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 228.865986] gitaly[1373]: time="2026-06-16T09:17:07.010Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGKRGAKHYVHRQQQ4T7P98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-06-16T09:17:36.399" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:06.999" grpc.time_ms=10.361 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 228.867171] gitaly[1373]: time="2026-06-16T09:17:07.182Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGQNQ7T1PKBEP9DPK6SV2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:07.172" grpc.time_ms=10.285 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 228.883758] gitaly[1373]: time="2026-06-16T09:17:07.254Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGQQJS26NHK15VWJXYKAA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:07.238" pid=1373 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 228.900205] gitaly[1373]: time="2026-06-16T09:17:07.254Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGQQJS26NHK15VWJXYKAA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:07.238" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 228.970236] gitaly[1373]: time="2026-06-16T09:17:07.254Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGQQJS26NHK15VWJXYKAA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:07.238" grpc.time_ms=16.266 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 228.971811] gitaly[1373]: time="2026-06-16T09:17:07.327Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGQSVN8NRZDPW10XTP9ZX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:07.305" pid=1373 remote_ip="2001:db8:1::1" request_sha=fcaf453774c25cfac836082ecfe0499a21fa854a response_bytes=383 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 228.972950] gitaly[1373]: time="2026-06-16T09:17:07.327Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1202 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGQSVN8NRZDPW10XTP9ZX grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:07.305" negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 228.980971] gitaly[1373]: time="2026-06-16T09:17:07.327Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1202 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGQSVN8NRZDPW10XTP9ZX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-06-16T09:17:07.305" grpc.time_ms=21.621 negotiation.command=fetch negotiation.haves=1 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 229.053609] gitaly[1373]: time="2026-06-16T09:17:07.675Z" level=info msg="finished unary call with code OK" catfile.duration_ms=95 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=95 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGQX80H63KBAA3FAB9RX2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:37.378" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:07.578" grpc.time_ms=96.169 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.055150] gitaly[1373]: time="2026-06-16T09:17:07.714Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGQX80H63KBAA3FAB9RX2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-06-16T09:17:37.505" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:07.706" grpc.time_ms=7.855 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.055682] gitaly[1373]: time="2026-06-16T09:17:07.732Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGQX80H63KBAA3FAB9RX2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-16T09:17:17.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:07.723" grpc.time_ms=8.549 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.067340] gitaly[1373]: time="2026-06-16T09:17:07.785Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGQX80H63KBAA3FAB9RX2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-06-16T09:17:17.064" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:17:07.785" grpc.time_ms=0.272 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.084151] gitaly[1373]: time="2026-06-16T09:17:07.797Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGQX80H63KBAA3FAB9RX2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-06-16T09:17:17.069" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:07.790" grpc.time_ms=6.947 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.166325] gitaly[1373]: time="2026-06-16T09:17:07.891Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGRCA4R4T51XVMGEH2Z39 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:37.689" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:07.889" grpc.time_ms=1.167 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.167813] gitaly[1373]: time="2026-06-16T09:17:08.414Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGQX80H63KBAA3FAB9RX2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:08.413" grpc.time_ms=0.417 pid=1373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 229.168734] gitaly[1373]: time="2026-06-16T09:17:08.441Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGQX80H63KBAA3FAB9RX2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:08.439" grpc.time_ms=2.22 pid=1373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 229.169838] gitaly[1373]: time="2026-06-16T09:17:08.452Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGQX80H63KBAA3FAB9RX2 diskcache=3a39fdf6-d2cd-4979-acd9-9e0e2b6ebf32 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-16T09:17:17.079" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:17:07.800" pid=1373 ref=refs/keep-around/f236896cb5377ee51df845bb2a34d4215b03d088 remote_ip="2001:db8:1::1" revision=f236896cb5377ee51df845bb2a34d4215b03d088 span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.173924] gitaly[1373]: time="2026-06-16T09:17:08.452Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=126 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=856064 command.minflt=5333 command.oublock=8 command.real_time_ms=643 command.spawn_token_fork_ms=0 command.system_time_ms=88 command.user_time_ms=38 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGQX80H63KBAA3FAB9RX2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-06-16T09:17:17.079" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-06-16T09:17:07.800" grpc.time_ms=652.782 pid=1373 ref=refs/keep-around/f236896cb5377ee51df845bb2a34d4215b03d088 remote_ip="2001:db8:1::1" revision=f236896cb5377ee51df845bb2a34d4215b03d088 span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.198150] gitaly[1373]: time="2026-06-16T09:17:08.457Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGQX80H63KBAA3FAB9RX2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:38.256" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:08.456" grpc.time_ms=0.471 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.218434] gitaly[1373]: time="2026-06-16T09:17:08.638Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGQX80H63KBAA3FAB9RX2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-16T09:17:38.037" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:08.637" grpc.time_ms=0.831 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.239294] gitaly[1373]: time="2026-06-16T09:17:08.665Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGQX80H63KBAA3FAB9RX2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:38.064" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:08.665" grpc.time_ms=0.578 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.312607] gitaly[1373]: time="2026-06-16T09:17:08.695Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGQX80H63KBAA3FAB9RX2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:38.094" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:08.694" grpc.time_ms=0.596 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.314148] gitaly[1373]: time="2026-06-16T09:17:08.983Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGSEM05BBTY39Z45B30VY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:38.782" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:08.982" grpc.time_ms=0.658 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.314794] gitaly[1373]: time="2026-06-16T09:17:10.080Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGTGJ8DAW6TECGA0ZDS95 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:40.078" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:10.079" grpc.time_ms=0.703 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.318196] gitaly[1373]: time="2026-06-16T09:17:11.179Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGVK2E6BMPNA0N4K43GE3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:41.078" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:11.178" grpc.time_ms=0.609 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses vm-test-run-nixbot-gitlab> gitlab # [ 229.339208] gitaly[1373]: time="2026-06-16T09:17:12.271Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGWNBXF4Z0Y22VSMF2HG3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:42.070" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:12.270" grpc.time_ms=0.859 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.433904] nixbot[3228]: {"ts": "2026-06-16T09:17:13.316Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-f236896cb537-fcc74a9c", "argv0": "bwrap"} vm-test-run-nixbot-gitlab> gitlab # [ 229.443799] nix-daemon[3498]: unexpected Nix daemon error: error: interrupted by the user vm-test-run-nixbot-gitlab> gitlab # [ 229.511945] gitaly[1373]: time="2026-06-16T09:17:13.394Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGXQYWVHFQWAADBH8T9NP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:43.393" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:13.393" grpc.time_ms=0.905 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.521702] nix-daemon[3491]: accepted connection from pid 3626, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 229.539924] nix-daemon[3491]: accepted connection from pid 3625, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 229.549875] gitaly[1373]: time="2026-06-16T09:17:13.432Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGXQYWVHFQWAADBH8T9NP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-16T09:17:23.015" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:13.425" grpc.time_ms=7.015 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.612909] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:13.495Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"bc46349bba450fc412404b4c","created_at":"2026-06-16T09:17:13.489Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-c896bd733e2a43b2","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.493Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-bc46349bba450fc412404b4c: start","job_status":"start","queue_duration_s":0.00151,"scheduling_latency_s":0.001552} vm-test-run-nixbot-gitlab> gitlab # [ 229.618869] gitaly[1373]: time="2026-06-16T09:17:13.501Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGXQXG3Y86SD56M64579A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:43.300" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:13.501" grpc.time_ms=0.575 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.627239] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:13 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf HTTP/1.1" 201 651 "" "python-httpx/0.28.1" 167 vm-test-run-nixbot-gitlab> gitlab # [ 229.629187] nixbot[3228]: {"ts": "2026-06-16T09:17:13.511Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 229.678532] gitaly[1373]: time="2026-06-16T09:17:13.561Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGXXD9Q17B913WR1GQ1CS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:43.559" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:13.560" grpc.time_ms=0.688 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.700079] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:13 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 241 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses, in 0.29 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-16T09:17:08.562Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 229.715782] gitaly[1373]: time="2026-06-16T09:17:13.598Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGXXD9Q17B913WR1GQ1CS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-16T09:17:23.180" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:13.591" grpc.time_ms=6.869 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.741786] gitaly[1373]: time="2026-06-16T09:17:13.624Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGXQYWVHFQWAADBH8T9NP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:13.624" grpc.time_ms=0.323 pid=1373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 229.756069] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:13 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf HTTP/1.1" 201 665 "" "python-httpx/0.28.1" 121 vm-test-run-nixbot-gitlab> gitlab # [ 229.757168] nixbot[3228]: {"ts": "2026-06-16T09:17:13.639Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ed9f2d2dd3145d3d3cb79c30ce31ece1faf70acf \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 229.790761] gitaly[1373]: time="2026-06-16T09:17:13.673Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGXQYWVHFQWAADBH8T9NP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:13.672" grpc.time_ms=0.705 pid=1373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 229.828556] gitaly[1373]: time="2026-06-16T09:17:13.711Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VGXQYWVHFQWAADBH8T9NP grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:13.710" grpc.time_ms=0.508 pid=1373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 229.858363] gitaly[1373]: time="2026-06-16T09:17:13.741Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGXQYWVHFQWAADBH8T9NP diskcache=926ca33d-8a80-4125-8465-c6e96819c4be grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-16T09:17:43.491" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:13.592" pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.874441] gitaly[1373]: time="2026-06-16T09:17:13.757Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=103 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7331 command.oublock=0 command.real_time_ms=138 command.spawn_token_fork_ms=0 command.system_time_ms=71 command.user_time_ms=32 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGXQYWVHFQWAADBH8T9NP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-16T09:17:43.491" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:13.592" grpc.time_ms=149.375 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 229.974776] gitaly[1373]: time="2026-06-16T09:17:13.857Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGXQYWVHFQWAADBH8T9NP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-16T09:17:43.255" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:13.855" grpc.time_ms=1.637 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 230.082562] gitaly[1373]: time="2026-06-16T09:17:13.964Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGXQYWVHFQWAADBH8T9NP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:43.361" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:13.962" grpc.time_ms=1.647 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 230.286279] gitaly[1373]: time="2026-06-16T09:17:14.168Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGXQYWVHFQWAADBH8T9NP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:44.067" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:14.167" grpc.time_ms=1.106 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 230.333843] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.185Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"db09eb8235860e0287dee3f6","created_at":"2026-06-16T09:17:14.181Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"bc46349bba450fc412404b4c","job_size_bytes":3,"pid":2167,"job_status":"deduplicated","message":"PipelineProcessWorker JID-db09eb8235860e0287dee3f6: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} vm-test-run-nixbot-gitlab> gitlab # [ 230.363890] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.188Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"bc46349bba450fc412404b4c","created_at":"2026-06-16T09:17:13.489Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-c896bd733e2a43b2","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.493Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2167,"job_status":"rescheduled","message":"PipelineProcessWorker JID-bc46349bba450fc412404b4c: rescheduled"} vm-test-run-nixbot-gitlab> gitlab # [ 230.387883] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.194Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"bc46349bba450fc412404b4c","created_at":"2026-06-16T09:17:13.489Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-c896bd733e2a43b2","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.493Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-bc46349bba450fc412404b4c: done: 0.699914 sec","job_status":"done","queue_duration_s":0.00151,"scheduling_latency_s":0.001552,"gitaly_calls":4,"gitaly_duration_s":0.229313,"redis_calls":104,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.040352,"redis_read_bytes":264,"redis_write_bytes":24006,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001386,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":17,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003778,"redis_cache_read_bytes":33,"redis_cache_write_bytes":1526,"redis_feature_flag_calls":22,"redis_feature_flag_duration_s":0.011675,"redis_feature_flag_read_bytes":22,"redis_feature_flag_write_bytes":3871,"redis_queues_calls":22,"redis_queues_duration_s":0.006211,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12385,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.011113,"redis_queues_metadata_read_bytes":110,"redis_queues_metadata_write_bytes":3546,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000468,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000835,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0035,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":850,"db_count":47,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.185,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.011,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.014,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.157,"db_ci_duration_s":0.028,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.250729,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.655487499,"duration_s":0.699914,"completed_at":"2026-06-16T09:17:14.194Z","load_balancing_strategy":"primary","db_duration_s":0.190648,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 230.401459] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.196Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f9b1fea7dd49e84707202ac5","created_at":"2026-06-16T09:17:13.573Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.578Z","job_size_bytes":23,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f9b1fea7dd49e84707202ac5: start","job_status":"start","queue_duration_s":0.617992,"scheduling_latency_s":0.618017} vm-test-run-nixbot-gitlab> gitlab # [ 230.479748] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.337Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"f9b1fea7dd49e84707202ac5","created_at":"2026-06-16T09:17:13.573Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.578Z","job_size_bytes":23,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-f9b1fea7dd49e84707202ac5: done: 0.140488 sec","job_status":"done","queue_duration_s":0.617992,"scheduling_latency_s":0.618017,"redis_calls":15,"redis_duration_s":0.007575,"redis_read_bytes":13,"redis_write_bytes":3012,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000895,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_calls":2,"redis_queues_duration_s":0.000664,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1627,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003654,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":742,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.001389,"redis_rate_limiting_write_bytes":148,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.107963,"worker_id":"sidekiq_0","rate_limiting_gates":["permanent_email_failure","temporary_email_failure"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00097666,"duration_s":0.140488,"completed_at":"2026-06-16T09:17:14.337Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021626,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 230.505662] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.339Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"9f66974a139326c395b659d9","created_at":"2026-06-16T09:17:14.326Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-c3e63912d63a44c9","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:1a37e6987b2d7b139ec92b9baaafe1df6157131bf93e146b9ce903a01f55fb43","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:14.329Z","job_size_bytes":464,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-9f66974a139326c395b659d9: start","job_status":"start","queue_duration_s":0.010029,"scheduling_latency_s":0.010053} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses vm-test-run-nixbot-gitlab> gitlab # [ 230.577121] sidekiq-cluster[2167]: {"severity":"WARN","time":"2026-06-16T09:17:14.379Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"9f66974a139326c395b659d9","created_at":"2026-06-16T09:17:14.326Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-c3e63912d63a44c9","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineNotificationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:1a37e6987b2d7b139ec92b9baaafe1df6157131bf93e146b9ce903a01f55fb43","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:14.329Z","job_size_bytes":464,"pid":2167,"sidekiq_tid":"m1d3","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-9f66974a139326c395b659d9: fail: 0.040114 sec","job_status":"fail","queue_duration_s":0.010029,"scheduling_latency_s":0.010053,"redis_calls":7,"redis_duration_s":0.003202,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002128,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021833,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040114,"completed_at":"2026-06-16T09:17:14.379Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/pipelines.rb:33:in `pipeline_mail'","app/mailers/emails/pipelines.rb:10:in `pipeline_failed_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.010013,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 231.194078] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.383Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"93a2fe18cbc72b246434751f","created_at":"2026-06-16T09:17:13.763Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.767Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-93a2fe18cbc72b246434751f: start","job_status":"start","queue_duration_s":0.615158,"scheduling_latency_s":0.61521} vm-test-run-nixbot-gitlab> gitlab # [ 231.195429] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.545Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"93a2fe18cbc72b246434751f","created_at":"2026-06-16T09:17:13.763Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.767Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-93a2fe18cbc72b246434751f: done: 0.162656 sec","job_status":"done","queue_duration_s":0.615158,"scheduling_latency_s":0.61521,"redis_calls":10,"redis_duration_s":0.003509,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000779,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002027,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000703,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.114,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.111,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000633879,"duration_s":0.162656,"completed_at":"2026-06-16T09:17:14.545Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.114356,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 231.273224] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.548Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c962ba74b65a6ef6e7fdd3dd","created_at":"2026-06-16T09:17:13.780Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.786Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c962ba74b65a6ef6e7fdd3dd: start","job_status":"start","queue_duration_s":0.762235,"scheduling_latency_s":0.762286} vm-test-run-nixbot-gitlab> gitlab # [ 231.274759] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.675Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c962ba74b65a6ef6e7fdd3dd","created_at":"2026-06-16T09:17:13.780Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.786Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c962ba74b65a6ef6e7fdd3dd: done: 0.126958 sec","job_status":"done","queue_duration_s":0.762235,"scheduling_latency_s":0.762286,"redis_calls":10,"redis_duration_s":0.003745,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000748,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002295,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000702,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.091,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.09,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00062494,"duration_s":0.126958,"completed_at":"2026-06-16T09:17:14.675Z","load_balancing_strategy":"primary","db_duration_s":0.091279,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 231.328865] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.677Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8792b2ae51ab3ca5be1acad3","created_at":"2026-06-16T09:17:13.803Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.810Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8792b2ae51ab3ca5be1acad3: start","job_status":"start","queue_duration_s":0.867852,"scheduling_latency_s":0.867876} vm-test-run-nixbot-gitlab> gitlab # [ 231.417854] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.711Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8792b2ae51ab3ca5be1acad3","created_at":"2026-06-16T09:17:13.803Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.810Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8792b2ae51ab3ca5be1acad3: done: 0.033379 sec","job_status":"done","queue_duration_s":0.867852,"scheduling_latency_s":0.867876,"redis_calls":13,"redis_duration_s":0.005383,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000759,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002587,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001209,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000828,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000592813,"duration_s":0.033379,"completed_at":"2026-06-16T09:17:14.711Z","load_balancing_strategy":"primary","db_duration_s":0.00448,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 231.424137] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.714Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"353f5afa3a2658efe36ed95a","created_at":"2026-06-16T09:17:13.810Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.815Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-353f5afa3a2658efe36ed95a: start","job_status":"start","queue_duration_s":0.898361,"scheduling_latency_s":0.898447} vm-test-run-nixbot-gitlab> gitlab # [ 231.450954] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.747Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"353f5afa3a2658efe36ed95a","created_at":"2026-06-16T09:17:13.810Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.815Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-353f5afa3a2658efe36ed95a: done: 0.033306 sec","job_status":"done","queue_duration_s":0.898361,"scheduling_latency_s":0.898447,"redis_calls":10,"redis_duration_s":0.004135,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000823,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002457,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000855,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000557334,"duration_s":0.033306,"completed_at":"2026-06-16T09:17:14.747Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006458,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 231.543369] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.749Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ff5b9dacd5bef19deeea37bb","created_at":"2026-06-16T09:17:13.817Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.831Z","job_size_bytes":19,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ff5b9dacd5bef19deeea37bb: start","job_status":"start","queue_duration_s":0.917562,"scheduling_latency_s":0.917583} vm-test-run-nixbot-gitlab> gitlab # [ 231.546975] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.778Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ff5b9dacd5bef19deeea37bb","created_at":"2026-06-16T09:17:13.817Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.831Z","job_size_bytes":19,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ff5b9dacd5bef19deeea37bb: done: 0.029387 sec","job_status":"done","queue_duration_s":0.917562,"scheduling_latency_s":0.917583,"redis_calls":11,"redis_duration_s":0.004643,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000866,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002932,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000845,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000570185,"duration_s":0.029387,"completed_at":"2026-06-16T09:17:14.778Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005714,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 231.666336] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.780Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"4e22e9118a427f4d5eb32530","created_at":"2026-06-16T09:17:13.835Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.843Z","job_size_bytes":65,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-4e22e9118a427f4d5eb32530: start","job_status":"start","queue_duration_s":0.936817,"scheduling_latency_s":0.936839} vm-test-run-nixbot-gitlab> gitlab # [ 231.667847] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.876Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"failed\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"4e22e9118a427f4d5eb32530","created_at":"2026-06-16T09:17:13.835Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2e8f76273ed3dffe93170e4a8828cd9c1cde36f18ce9deaf751670cff2f2dad","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.843Z","job_size_bytes":65,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-4e22e9118a427f4d5eb32530: done: 0.096328 sec","job_status":"done","queue_duration_s":0.936817,"scheduling_latency_s":0.936839,"redis_calls":10,"redis_duration_s":0.005768,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000453,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003695,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00162,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001397105,"duration_s":0.096328,"completed_at":"2026-06-16T09:17:14.876Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006132,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 231.716320] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.883Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"461d96dd47a7a732b400f001","created_at":"2026-06-16T09:17:13.845Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.847Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-461d96dd47a7a732b400f001: start","job_status":"start","queue_duration_s":1.035639,"scheduling_latency_s":1.035689} vm-test-run-nixbot-gitlab> gitlab # [ 231.795318] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.917Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"461d96dd47a7a732b400f001","created_at":"2026-06-16T09:17:13.845Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.847Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-461d96dd47a7a732b400f001: done: 0.034423 sec","job_status":"done","queue_duration_s":1.035639,"scheduling_latency_s":1.035689,"redis_calls":7,"redis_duration_s":0.002833,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002147,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000686,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023183,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034423,"completed_at":"2026-06-16T09:17:14.917Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003223,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 231.807741] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.919Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"049e78948ffda3a516260885","created_at":"2026-06-16T09:17:13.966Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.978Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-049e78948ffda3a516260885: start","job_status":"start","queue_duration_s":0.941123,"scheduling_latency_s":0.941148} vm-test-run-nixbot-gitlab> gitlab # [ 231.834130] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.948Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"049e78948ffda3a516260885","created_at":"2026-06-16T09:17:13.966Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.978Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-049e78948ffda3a516260885: done: 0.029165 sec","job_status":"done","queue_duration_s":0.941123,"scheduling_latency_s":0.941148,"redis_calls":10,"redis_duration_s":0.002166,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001274,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000457,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016264,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000339708,"duration_s":0.029165,"completed_at":"2026-06-16T09:17:14.948Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006942,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 231.936370] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.950Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"1923007692216adb3f26bfd5","created_at":"2026-06-16T09:17:13.979Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.987Z","job_size_bytes":6,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-1923007692216adb3f26bfd5: start","job_status":"start","queue_duration_s":0.962208,"scheduling_latency_s":0.96223} vm-test-run-nixbot-gitlab> gitlab # [ 231.937897] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.970Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"1923007692216adb3f26bfd5","created_at":"2026-06-16T09:17:13.979Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:13.987Z","job_size_bytes":6,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-1923007692216adb3f26bfd5: done: 0.020275 sec","job_status":"done","queue_duration_s":0.962208,"scheduling_latency_s":0.96223,"redis_calls":10,"redis_duration_s":0.002326,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000468,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001381,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000477,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000365689,"duration_s":0.020275,"completed_at":"2026-06-16T09:17:14.970Z","load_balancing_strategy":"primary","db_duration_s":0.001631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 232.027053] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:14.971Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"692554025d1040d7acf845f4","created_at":"2026-06-16T09:17:14.188Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:14.189Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-692554025d1040d7acf845f4: start","job_status":"start","queue_duration_s":0.782022,"scheduling_latency_s":0.782043} vm-test-run-nixbot-gitlab> gitlab # [ 232.028852] gitaly[1373]: time="2026-06-16T09:17:15.016Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VGXQYWVHFQWAADBH8T9NP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-16T09:17:45.014" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:15.015" grpc.time_ms=0.857 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 232.097504] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:15.058Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"692554025d1040d7acf845f4","created_at":"2026-06-16T09:17:14.188Z","trace_propagation_headers":{"sentry-trace":"8f4825865f52496eb450eb96f996ab6b-92f0611bc0d74db6","baggage":"sentry-trace_id=8f4825865f52496eb450eb96f996ab6b,sentry-release=Unknown"},"correlation_id":"01KV7VGXQYWVHFQWAADBH8T9NP","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:14.189Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-692554025d1040d7acf845f4: done: 0.087144 sec","job_status":"done","queue_duration_s":0.782022,"scheduling_latency_s":0.782043,"gitaly_calls":2,"gitaly_duration_s":0.006396,"redis_calls":19,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.00407,"redis_read_bytes":42,"redis_write_bytes":1891,"redis_cache_calls":8,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000797,"redis_cache_read_bytes":16,"redis_cache_write_bytes":735,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001305,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000246,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001722,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":20,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":20,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":9,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.062748198,"duration_s":0.087144,"completed_at":"2026-06-16T09:17:15.058Z","load_balancing_strategy":"primary","db_duration_s":0.017588,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 232.114485] gitaly[1373]: time="2026-06-16T09:17:15.046Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VGXQYWVHFQWAADBH8T9NP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:45.045" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:15.045" grpc.time_ms=0.707 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 232.261728] gitaly[1373]: time="2026-06-16T09:17:16.144Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VH0E7J2PGYYBT2PBBXT80 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:46.142" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:16.143" grpc.time_ms=1.041 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 232.307401] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 85 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses, in 1.63 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-16T09:17:08.562Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses vm-test-run-nixbot-gitlab> gitlab # [ 233.441334] gitaly[1373]: time="2026-06-16T09:17:17.323Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VH1J4V021T982WS703J1C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:47.121" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:17.322" grpc.time_ms=0.666 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 233.471126] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 100 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses, in 0.17 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-06-16T09:17:08.562Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 234.054525] nix-daemon[3491]: accepted connection from pid 3680, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 234.069781] gitaly[1373]: time="2026-06-16T09:17:17.952Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VH268FFNQ8HBKWRP66TCJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:47.749" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:17.951" grpc.time_ms=1.19 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 234.081354] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:17.963Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"d941db8fbe9258fdaf7e9fb5","created_at":"2026-06-16T09:16:54.470Z","trace_propagation_headers":{"sentry-trace":"a1206a821b7b46cab787b3bbcb83607a-8475d8124ba34848","baggage":"sentry-trace_id=a1206a821b7b46cab787b3bbcb83607a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGAQBSKZRVYY46762XQM9","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:35ba3b039afb466fbba0192235e5d48a7add31112fab869c33bfc281eba7de8f","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:17.960Z","retry_count":0,"job_size_bytes":451,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-d941db8fbe9258fdaf7e9fb5: start","job_status":"start","queue_duration_s":0.002833,"scheduling_latency_s":0.002884} vm-test-run-nixbot-gitlab> gitlab # [ 234.179609] gitaly[1373]: time="2026-06-16T09:17:18.062Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KV7VH268FFNQ8HBKWRP66TCJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-16T09:17:28.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:18.053" grpc.time_ms=8.377 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 234.250805] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:18 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f236896cb5377ee51df845bb2a34d4215b03d088 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 236 vm-test-run-nixbot-gitlab> gitlab # [ 234.258216] nixbot[3228]: {"ts": "2026-06-16T09:17:18.140Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f236896cb5377ee51df845bb2a34d4215b03d088 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses, in 0.13 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-16T09:17:08.562Z', 'started_at': None, 'finished_at': '2026-06-16T09:17:18.093Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-16T09:17:18.407Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 234.261955] sidekiq-cluster[2167]: {"severity":"WARN","time":"2026-06-16T09:17:18.143Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"d941db8fbe9258fdaf7e9fb5","created_at":"2026-06-16T09:16:54.470Z","trace_propagation_headers":{"sentry-trace":"a1206a821b7b46cab787b3bbcb83607a-8475d8124ba34848","baggage":"sentry-trace_id=a1206a821b7b46cab787b3bbcb83607a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VGAQBSKZRVYY46762XQM9","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:35ba3b039afb466fbba0192235e5d48a7add31112fab869c33bfc281eba7de8f","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:17.960Z","retry_count":0,"job_size_bytes":451,"pid":2167,"sidekiq_tid":"m8en","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-d941db8fbe9258fdaf7e9fb5: fail: 0.179477 sec","job_status":"fail","queue_duration_s":0.002833,"scheduling_latency_s":0.002884,"redis_calls":7,"redis_duration_s":0.005795,"redis_read_bytes":5,"redis_write_bytes":786,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004313,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001482,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.134,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.134,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.179477,"completed_at":"2026-06-16T09:17:18.142Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.133699,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 234.814112] gitaly[1373]: time="2026-06-16T09:17:18.303Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VH268FFNQ8HBKWRP66TCJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:18.303" grpc.time_ms=0.508 pid=1373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 234.817054] gitaly[1373]: time="2026-06-16T09:17:18.353Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VH2K4112VQ1CEJQS14KT4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:48.352" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:18.352" grpc.time_ms=0.966 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 234.901105] nixbot[3228]: {"ts": "2026-06-16T09:17:18.428Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f236896cb5377ee51df845bb2a34d4215b03d088 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab # [ 234.902380] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:18.155Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f3bdaa6e75c93b27f8a50608","created_at":"2026-06-16T09:17:18.104Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-5224fc200e1c4cd8","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:18.111Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f3bdaa6e75c93b27f8a50608: start","job_status":"start","queue_duration_s":0.043699,"scheduling_latency_s":0.043734} vm-test-run-nixbot-gitlab> gitlab # [ 234.903639] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:18 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f236896cb5377ee51df845bb2a34d4215b03d088 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 117 vm-test-run-nixbot-gitlab> gitlab # [ 234.904378] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:18 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 74 vm-test-run-nixbot-gitlab> gitlab # [ 234.905316] gitaly[1373]: time="2026-06-16T09:17:18.387Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VH2K4112VQ1CEJQS14KT4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-16T09:17:28.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:18.376" grpc.time_ms=10.556 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 234.906285] gitaly[1373]: time="2026-06-16T09:17:18.481Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VH2Q3T8Q6Q0BN0N7SBTGK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:48.280" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:18.480" grpc.time_ms=0.792 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 235.000420] gitaly[1373]: time="2026-06-16T09:17:18.883Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VH268FFNQ8HBKWRP66TCJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:18.882" grpc.time_ms=0.471 pid=1373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 235.042646] gitaly[1373]: time="2026-06-16T09:17:18.925Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VH268FFNQ8HBKWRP66TCJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:18.924" grpc.time_ms=0.539 pid=1373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 235.068490] gitaly[1373]: time="2026-06-16T09:17:18.951Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KV7VH268FFNQ8HBKWRP66TCJ diskcache=4e916293-3267-4452-beb4-226f0960a8c5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-16T09:17:48.263" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:18.264" pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 235.069932] gitaly[1373]: time="2026-06-16T09:17:18.951Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=137 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=819200 command.minflt=6663 command.oublock=0 command.real_time_ms=677 command.spawn_token_fork_ms=1 command.system_time_ms=100 command.user_time_ms=37 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VH268FFNQ8HBKWRP66TCJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-06-16T09:17:48.263" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:18.264" grpc.time_ms=686.743 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 235.147741] gitaly[1373]: time="2026-06-16T09:17:19.030Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VH268FFNQ8HBKWRP66TCJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-16T09:17:49.028" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:19.029" grpc.time_ms=0.859 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 235.186046] gitaly[1373]: time="2026-06-16T09:17:19.068Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VH268FFNQ8HBKWRP66TCJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:49.067" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:19.068" grpc.time_ms=0.593 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 235.248279] gitaly[1373]: time="2026-06-16T09:17:19.131Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VH268FFNQ8HBKWRP66TCJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:49.129" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:19.130" grpc.time_ms=0.907 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 235.287675] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.169Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"1f102341b02516295b44ae89","created_at":"2026-06-16T09:17:19.163Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"f3bdaa6e75c93b27f8a50608","job_size_bytes":3,"pid":2167,"job_status":"deduplicated","message":"PipelineProcessWorker JID-1f102341b02516295b44ae89: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} vm-test-run-nixbot-gitlab> gitlab # [ 235.312699] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.174Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f3bdaa6e75c93b27f8a50608","created_at":"2026-06-16T09:17:18.104Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-5224fc200e1c4cd8","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:18.111Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2167,"job_status":"rescheduled","message":"PipelineProcessWorker JID-f3bdaa6e75c93b27f8a50608: rescheduled"} vm-test-run-nixbot-gitlab> gitlab # [ 235.313805] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.186Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f3bdaa6e75c93b27f8a50608","created_at":"2026-06-16T09:17:18.104Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-5224fc200e1c4cd8","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:18.111Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f3bdaa6e75c93b27f8a50608: done: 1.031453 sec","job_status":"done","queue_duration_s":0.043699,"scheduling_latency_s":0.043734,"gitaly_calls":4,"gitaly_duration_s":0.701299,"redis_calls":93,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.08608299999999999,"redis_read_bytes":262,"redis_write_bytes":23390,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.004076,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.026401,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003064,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":712,"redis_queues_calls":25,"redis_queues_duration_s":0.009885,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14471,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.025816,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.001172,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.003134,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.008459,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":38,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":38,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.07,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":23,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.013,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.034,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.150427,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.966806954,"duration_s":1.031453,"completed_at":"2026-06-16T09:17:19.186Z","load_balancing_strategy":"primary","db_duration_s":0.074901,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 235.350445] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.190Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"bcdcbe179558a486e4b9268d","created_at":"2026-06-16T09:17:18.249Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:18.255Z","job_size_bytes":23,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-bcdcbe179558a486e4b9268d: start","job_status":"start","queue_duration_s":0.935104,"scheduling_latency_s":0.935135} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses vm-test-run-nixbot-gitlab> gitlab # [ 235.560457] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.442Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"bcdcbe179558a486e4b9268d","created_at":"2026-06-16T09:17:18.249Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:18.255Z","job_size_bytes":23,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-bcdcbe179558a486e4b9268d: done: 0.252069 sec","job_status":"done","queue_duration_s":0.935104,"scheduling_latency_s":0.935135,"redis_calls":7,"redis_duration_s":0.016848000000000002,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.01111,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005738,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.252069,"completed_at":"2026-06-16T09:17:19.442Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027007,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 235.664891] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.547Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"62bad33838d744118b37e188","created_at":"2026-06-16T09:17:18.959Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:18.963Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-62bad33838d744118b37e188: start","job_status":"start","queue_duration_s":0.583313,"scheduling_latency_s":0.583389} vm-test-run-nixbot-gitlab> gitlab # [ 235.762449] gitaly[1373]: time="2026-06-16T09:17:19.645Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VH3TEFPZXMP07FHTHKRFK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:49.642" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:19.643" grpc.time_ms=1.125 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 235.782838] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.665Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"62bad33838d744118b37e188","created_at":"2026-06-16T09:17:18.959Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:18.963Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-62bad33838d744118b37e188: done: 0.118449 sec","job_status":"done","queue_duration_s":0.583313,"scheduling_latency_s":0.583389,"redis_calls":7,"redis_duration_s":0.014372,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008866,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005506,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.118449,"completed_at":"2026-06-16T09:17:19.665Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009487,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 235.791499] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.674Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7617d515bde5276fcd63e9cd","created_at":"2026-06-16T09:17:18.977Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:18.981Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7617d515bde5276fcd63e9cd: start","job_status":"start","queue_duration_s":0.692867,"scheduling_latency_s":0.692906} vm-test-run-nixbot-gitlab> gitlab # [ 235.905940] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:19 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 220 vm-test-run-nixbot-gitlab> gitlab # [ 235.910236] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.792Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7617d515bde5276fcd63e9cd","created_at":"2026-06-16T09:17:18.977Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:18.981Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7617d515bde5276fcd63e9cd: done: 0.11865 sec","job_status":"done","queue_duration_s":0.692867,"scheduling_latency_s":0.692906,"redis_calls":7,"redis_duration_s":0.007181999999999999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004501,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002681,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.061,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.061,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.11865,"completed_at":"2026-06-16T09:17:19.792Z","load_balancing_strategy":"primary","db_duration_s":0.06109,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 235.915756] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.798Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"367ed4dd1de3f8007a5fdf5b","created_at":"2026-06-16T09:17:18.996Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.000Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-367ed4dd1de3f8007a5fdf5b: start","job_status":"start","queue_duration_s":0.797837,"scheduling_latency_s":0.797868} vm-test-run-nixbot-gitlab> gitlab # [ 235.963125] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.845Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"367ed4dd1de3f8007a5fdf5b","created_at":"2026-06-16T09:17:18.996Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.000Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-367ed4dd1de3f8007a5fdf5b: done: 0.047126 sec","job_status":"done","queue_duration_s":0.797837,"scheduling_latency_s":0.797868,"redis_calls":10,"redis_duration_s":0.010513,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005324,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.0027,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002489,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047126,"completed_at":"2026-06-16T09:17:19.845Z","load_balancing_strategy":"primary","db_duration_s":0.005732,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 236.036122] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.850Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"de96cd334be7870e5083d20c","created_at":"2026-06-16T09:17:19.002Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.005Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-de96cd334be7870e5083d20c: start","job_status":"start","queue_duration_s":0.845086,"scheduling_latency_s":0.845119} vm-test-run-nixbot-gitlab> gitlab # [ 236.037362] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.886Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"de96cd334be7870e5083d20c","created_at":"2026-06-16T09:17:19.002Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.005Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-de96cd334be7870e5083d20c: done: 0.036416 sec","job_status":"done","queue_duration_s":0.845086,"scheduling_latency_s":0.845119,"redis_calls":7,"redis_duration_s":0.0073030000000000005,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004329,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002974,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036416,"completed_at":"2026-06-16T09:17:19.886Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005906,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 236.112439] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.891Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"cc44a63876a43a869611c0ac","created_at":"2026-06-16T09:17:19.006Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.010Z","job_size_bytes":19,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-cc44a63876a43a869611c0ac: start","job_status":"start","queue_duration_s":0.880271,"scheduling_latency_s":0.880299} vm-test-run-nixbot-gitlab> gitlab # [ 236.119321] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.932Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"cc44a63876a43a869611c0ac","created_at":"2026-06-16T09:17:19.006Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.010Z","job_size_bytes":19,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-cc44a63876a43a869611c0ac: done: 0.040963 sec","job_status":"done","queue_duration_s":0.880271,"scheduling_latency_s":0.880299,"redis_calls":8,"redis_duration_s":0.00807,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005433,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002637,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015685,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040963,"completed_at":"2026-06-16T09:17:19.932Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006134,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 236.211247] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:19.936Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"1efaadd70fefbc109591086a","created_at":"2026-06-16T09:17:19.012Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.015Z","job_size_bytes":66,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-1efaadd70fefbc109591086a: start","job_status":"start","queue_duration_s":0.92137,"scheduling_latency_s":0.921404} vm-test-run-nixbot-gitlab> gitlab # [ 236.215594] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:20.079Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"1efaadd70fefbc109591086a","created_at":"2026-06-16T09:17:19.012Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.015Z","job_size_bytes":66,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-1efaadd70fefbc109591086a: done: 0.142418 sec","job_status":"done","queue_duration_s":0.92137,"scheduling_latency_s":0.921404,"redis_calls":7,"redis_duration_s":0.010766999999999999,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004617,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00615,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.077,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.077,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.142418,"completed_at":"2026-06-16T09:17:20.079Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.077291,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 236.309585] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:20.086Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"7c9481b38718218468d6ee98","created_at":"2026-06-16T09:17:19.018Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.021Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7c9481b38718218468d6ee98: start","job_status":"start","queue_duration_s":1.064889,"scheduling_latency_s":1.064923} vm-test-run-nixbot-gitlab> gitlab # [ 236.313237] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:20.172Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"7c9481b38718218468d6ee98","created_at":"2026-06-16T09:17:19.018Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.021Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7c9481b38718218468d6ee98: done: 0.08547 sec","job_status":"done","queue_duration_s":1.064889,"scheduling_latency_s":1.064923,"redis_calls":7,"redis_duration_s":0.009031,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006603,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002428,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022468,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.08547,"completed_at":"2026-06-16T09:17:20.172Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016311,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 236.403354] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:20.176Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"3c7293366f771890f4c7f364","created_at":"2026-06-16T09:17:19.081Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.085Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-3c7293366f771890f4c7f364: start","job_status":"start","queue_duration_s":1.091517,"scheduling_latency_s":1.09155} vm-test-run-nixbot-gitlab> gitlab # [ 236.405052] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:20.232Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"3c7293366f771890f4c7f364","created_at":"2026-06-16T09:17:19.081Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.085Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-3c7293366f771890f4c7f364: done: 0.056408 sec","job_status":"done","queue_duration_s":1.091517,"scheduling_latency_s":1.09155,"redis_calls":7,"redis_duration_s":0.009731,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00159,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013814,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.056408,"completed_at":"2026-06-16T09:17:20.232Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022934,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 236.490187] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:20.237Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"23174511014e856a0ab9fe84","created_at":"2026-06-16T09:17:19.085Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.088Z","job_size_bytes":6,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-23174511014e856a0ab9fe84: start","job_status":"start","queue_duration_s":1.148893,"scheduling_latency_s":1.148918} vm-test-run-nixbot-gitlab> gitlab # [ 236.496932] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:20.262Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"23174511014e856a0ab9fe84","created_at":"2026-06-16T09:17:19.085Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.088Z","job_size_bytes":6,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-23174511014e856a0ab9fe84: done: 0.025122 sec","job_status":"done","queue_duration_s":1.148893,"scheduling_latency_s":1.148918,"redis_calls":7,"redis_duration_s":0.006205,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00399,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002215,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025122,"completed_at":"2026-06-16T09:17:20.262Z","load_balancing_strategy":"primary","db_duration_s":0.001124,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 236.571366] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:20.266Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"5e6aa7bc4e50edb335b44e77","created_at":"2026-06-16T09:17:19.089Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.096Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-5e6aa7bc4e50edb335b44e77: start","job_status":"start","queue_duration_s":1.170123,"scheduling_latency_s":1.170147} vm-test-run-nixbot-gitlab> gitlab # [ 236.588124] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:20.314Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"5e6aa7bc4e50edb335b44e77","created_at":"2026-06-16T09:17:19.089Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.096Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-5e6aa7bc4e50edb335b44e77: done: 0.048341 sec","job_status":"done","queue_duration_s":1.170123,"scheduling_latency_s":1.170147,"redis_calls":10,"redis_duration_s":0.008227,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001326,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004457,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002444,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000685842,"duration_s":0.048341,"completed_at":"2026-06-16T09:17:20.314Z","load_balancing_strategy":"primary","db_duration_s":0.00974,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 236.677256] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:20.319Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"8c99af973bc8a0d103e74cd6","created_at":"2026-06-16T09:17:19.175Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.177Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8c99af973bc8a0d103e74cd6: start","job_status":"start","queue_duration_s":1.141783,"scheduling_latency_s":1.141807} vm-test-run-nixbot-gitlab> gitlab # [ 236.746400] gitaly[1373]: time="2026-06-16T09:17:20.414Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KV7VH268FFNQ8HBKWRP66TCJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-06-16T09:17:50.413" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:20.413" grpc.time_ms=1.127 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 236.747780] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:20.546Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"8c99af973bc8a0d103e74cd6","created_at":"2026-06-16T09:17:19.175Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-6ceb6ad0c7084a30","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:19.177Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8c99af973bc8a0d103e74cd6: done: 0.226827 sec","job_status":"done","queue_duration_s":1.141783,"scheduling_latency_s":1.141807,"gitaly_calls":3,"gitaly_duration_s":0.014037,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.027763,"redis_read_bytes":98,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00363,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.005159,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000481,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.007091,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.001559,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.003881,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002332,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.081,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.053,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.096721,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.182448887,"duration_s":0.226827,"completed_at":"2026-06-16T09:17:20.546Z","load_balancing_strategy":"primary","db_duration_s":0.088841,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} vm-test-run-nixbot-gitlab> gitlab # [ 236.833129] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:20.550Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"7c3bf7c80b2a902d5d698b57","created_at":"2026-06-16T09:17:20.397Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-fa76317eb9c44888","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:20.401Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7c3bf7c80b2a902d5d698b57: start","job_status":"start","queue_duration_s":0.148665,"scheduling_latency_s":0.148692} vm-test-run-nixbot-gitlab> gitlab # [ 236.834419] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:20.598Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"7c3bf7c80b2a902d5d698b57","created_at":"2026-06-16T09:17:20.397Z","trace_propagation_headers":{"sentry-trace":"2d4f6397ffcf45ee8e3d026ea08f01bb-fa76317eb9c44888","baggage":"sentry-trace_id=2d4f6397ffcf45ee8e3d026ea08f01bb,sentry-release=Unknown"},"correlation_id":"01KV7VH268FFNQ8HBKWRP66TCJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:20.401Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7c3bf7c80b2a902d5d698b57: done: 0.048345 sec","job_status":"done","queue_duration_s":0.148665,"scheduling_latency_s":0.148692,"redis_calls":7,"redis_duration_s":0.007791,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005511,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00228,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048345,"completed_at":"2026-06-16T09:17:20.598Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006242,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} vm-test-run-nixbot-gitlab> gitlab # [ 236.917417] gitaly[1373]: time="2026-06-16T09:17:20.449Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VH268FFNQ8HBKWRP66TCJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:50.447" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:20.448" grpc.time_ms=1.144 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 236.918832] gitaly[1373]: time="2026-06-16T09:17:20.523Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VH268FFNQ8HBKWRP66TCJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:50.520" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:20.521" grpc.time_ms=0.745 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses, in 1.37 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-16T09:17:08.562Z', 'started_at': None, 'finished_at': '2026-06-16T09:17:18.093Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-16T09:17:18.407Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses vm-test-run-nixbot-gitlab> gitlab # [ 238.118648] gitaly[1373]: time="2026-06-16T09:17:22.000Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VH646ZJDD6Z39AKB31ZMK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:51.098" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:21.999" grpc.time_ms=1.339 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 238.212590] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 167 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses, in 0.22 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-16T09:17:08.562Z', 'started_at': None, 'finished_at': '2026-06-16T09:17:18.093Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-06-16T09:17:18.407Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> gitlab # [ 238.885888] nix-daemon[3491]: accepted connection from pid 3751, user nixbot vm-test-run-nixbot-gitlab> gitlab # [ 239.009226] gitaly[1373]: time="2026-06-16T09:17:22.891Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VH6ZVCG5VM2MYVKWGJ0K4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:52.889" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:22.890" grpc.time_ms=1.16 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 239.082766] gitaly[1373]: time="2026-06-16T09:17:22.965Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KV7VH6ZVCG5VM2MYVKWGJ0K4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-06-16T09:17:32.229" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-06-16T09:17:22.949" grpc.time_ms=15.64 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 239.144887] sidekiq-cluster[2167]: {"severity":"INFO","time":"2026-06-16T09:17:23.026Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f4899a0dc51500abda326b56","created_at":"2026-06-16T09:17:23.018Z","trace_propagation_headers":{"sentry-trace":"26abdfebcf0f4ddc825bcce5cf3510f7-5e0ab8648f39405a","baggage":"sentry-trace_id=26abdfebcf0f4ddc825bcce5cf3510f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KV7VH6ZVCG5VM2MYVKWGJ0K4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-06-16T09:17:23.022Z","job_size_bytes":3,"pid":2167,"sidekiq_tid":"m8ln","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f4899a0dc51500abda326b56: start","job_status":"start","queue_duration_s":0.003944,"scheduling_latency_s":0.003987} vm-test-run-nixbot-gitlab> gitlab # [ 239.178517] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:23 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/f236896cb5377ee51df845bb2a34d4215b03d088 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 234 vm-test-run-nixbot-gitlab> gitlab # [ 239.181332] nixbot[3228]: {"ts": "2026-06-16T09:17:23.047Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/f236896cb5377ee51df845bb2a34d4215b03d088 \"HTTP/1.1 201 Created\""} vm-test-run-nixbot-gitlab> gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses vm-test-run-nixbot-gitlab> gitlab # [ 239.286992] gitaly[1373]: time="2026-06-16T09:17:23.169Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VH6ZVCG5VM2MYVKWGJ0K4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:23.168" grpc.time_ms=0.515 pid=1373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 239.297448] gitaly[1373]: time="2026-06-16T09:17:23.180Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KV7VH7A2C1QRX948833DZPK8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-06-16T09:17:53.079" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-06-16T09:17:23.179" grpc.time_ms=0.676 pid=1373 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root vm-test-run-nixbot-gitlab> gitlab # [ 239.323849] gitlab-workhorse[1288]: gitlab 2001:db8:1::1 - - [2026/06/16:09:17:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 67 vm-test-run-nixbot-gitlab> gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/f236896cb5377ee51df845bb2a34d4215b03d088/statuses, in 0.11 seconds) vm-test-run-nixbot-gitlab> [{'id': 2, 'sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-06-16T09:17:08.562Z', 'started_at': None, 'finished_at': '2026-06-16T09:17:18.093Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 4, 'sha': 'f236896cb5377ee51df845bb2a34d4215b03d088', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-06-16T09:17:18.407Z', 'started_at': None, 'finished_at': '2026-06-16T09:17:23.009Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] vm-test-run-nixbot-gitlab> (finished: subtest: eval and build post commit statuses, in 15.38 seconds) vm-test-run-nixbot-gitlab> (finished: run the VM test script, in 274.84 seconds) vm-test-run-nixbot-gitlab> gitlab # [ 239.360250] gitaly[1373]: time="2026-06-16T09:17:23.243Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KV7VH6ZVCG5VM2MYVKWGJ0K4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-06-16T09:17:23.240" grpc.time_ms=2.82 pid=1373 span.kind=server system=grpc vm-test-run-nixbot-gitlab> gitlab # [ 239.368698] nix-daemon[3491]: accepted connection from pid 3782, user nixbot vm-test-run-nixbot-gitlab> test script finished in 274.89s vm-test-run-nixbot-gitlab> cleanup vm-test-run-nixbot-gitlab> kill QemuMachine (pid 13) vm-test-run-nixbot-gitlab> gitlab # qemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13) vm-test-run-nixbot-gitlab> (finished: cleanup, in 0.02 seconds) post-build step Upload coverage to codecov: ok Skipping codecov: project=Mic92/nixbot attr=x86_64-linux.nixbot-gitlab