Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh start all VMs gitlab: starting vm gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab # Formatting '/build/vm-state-gitlab/tmp.ijPTl92fW2', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) gitlab # Discarding device blocks: 0/262144 done gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: 66006670-0cbd-4e4d-afaf-d30e1d531023 gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab: QEMU running (pid 46) (finished: start all VMs, in 83.14 seconds) gitlab: waiting for unit setup-gitlab.service gitlab: waiting for the VM to finish booting gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... ok gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zifp21wm254bs0qi6zlmc55z81l1bgph-nixos-system-gitlab-test/init regInfo=/nix/store/mvhlh9k42b745pzdc7y47wxvdgdd1ckk-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] x86 CPU feature dependency check failure: CPU0 has '18*32+31' enabled but '18*32+26' disabled. Kernel might be fine, but no guarantees. gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x400000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000001] kvm-clock: using sched offset of 531627841 cycles gitlab # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000008] tsc: Detected 3099.998 MHz processor gitlab # [ 0.000747] last_pfn = 0x1c0000 max_arch_pfn = 0x400000000 gitlab # [ 0.000774] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000777] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000818] last_pfn = 0xbffd8 max_arch_pfn = 0x400000000 gitlab # [ 0.002871] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002888] Using GB pages for direct mapping gitlab # [ 0.002996] RAMDISK: [mem 0xbe31c000-0xbffcffff] gitlab # [ 0.003002] ACPI: Early table checksum verification disabled gitlab # [ 0.003006] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.003009] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.003015] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.003021] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.003024] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.003026] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.003028] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.003030] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.003031] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.003033] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.003033] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.003034] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.003035] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.003035] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003438] No NUMA configuration found gitlab # [ 0.003439] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003442] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff] gitlab # [ 0.003538] Zone ranges: gitlab # [ 0.003538] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.003540] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.003541] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.003542] Device empty gitlab # [ 0.003543] Movable zone start for each node gitlab # [ 0.003543] Early memory node ranges gitlab # [ 0.003544] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.003545] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.003546] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.003547] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.003567] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.003810] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.107159] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.107955] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.107971] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.108000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.108003] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.108005] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.108006] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.108007] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.108007] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.108010] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.108011] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.108017] TSC deadline timer available gitlab # [ 0.108023] CPU topo: Max. logical packages: 1 gitlab # [ 0.108023] CPU topo: Max. logical dies: 1 gitlab # [ 0.108024] CPU topo: Max. dies per package: 1 gitlab # [ 0.108028] CPU topo: Max. threads per core: 1 gitlab # [ 0.108028] CPU topo: Num. cores per package: 4 gitlab # [ 0.108029] CPU topo: Num. threads per package: 4 gitlab # [ 0.108029] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.108048] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.108064] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.108066] kvm-guest: setup PV sched yield gitlab # [ 0.108086] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.108088] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.108089] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.108090] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.108092] Booting paravirtualized kernel on KVM gitlab # [ 0.108096] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.112532] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.114948] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.115000] kvm-guest: PV spinlocks enabled gitlab # [ 0.115002] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.115004] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zifp21wm254bs0qi6zlmc55z81l1bgph-nixos-system-gitlab-test/init regInfo=/nix/store/mvhlh9k42b745pzdc7y47wxvdgdd1ckk-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.115101] Unknown kernel command line parameters "regInfo=/nix/store/mvhlh9k42b745pzdc7y47wxvdgdd1ckk-closure-info/registration", will be passed to user space. gitlab # [ 0.115120] random: crng init done gitlab # [ 0.115120] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.123534] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.127581] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.127663] software IO TLB: area num 4. gitlab # [ 0.203176] Fallback order for Node 0: 0 gitlab # [ 0.203188] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.203191] Policy zone: Normal gitlab # [ 0.205806] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.219109] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.231862] allocated 12582912 bytes of page_ext gitlab # [ 0.240243] ftrace: allocating 48717 entries in 192 pages gitlab # [ 0.240250] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.241206] Dynamic Preempt: lazy gitlab # [ 0.241451] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.241452] rcu: RCU event tracing is enabled. gitlab # [ 0.241452] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.241454] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.241454] Rude variant of Tasks RCU enabled. gitlab # [ 0.241454] Tracing variant of Tasks RCU enabled. gitlab # [ 0.241455] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.241456] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.241482] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.241484] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.241486] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.246220] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.246512] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.246520] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.246630] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.252597] Console: colour VGA+ 80x25 gitlab # [ 0.252603] printk: legacy console [tty0] enabled gitlab # [ 0.291640] printk: legacy console [ttyS0] enabled gitlab # [ 0.447589] ACPI: Core revision 20250807 gitlab # [ 0.448936] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.451348] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.452806] x2apic enabled gitlab # [ 0.453853] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.455274] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.457008] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.459050] kvm-guest: setup PV IPIs gitlab # [ 0.460919] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.462306] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x2caf45e6a57, max_idle_ns: 440795365367 ns gitlab # [ 0.464650] Calibrating delay loop (skipped) preset value.. 6199.99 BogoMIPS (lpj=3099998) gitlab # [ 0.467739] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.468795] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.469647] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.471655] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.472649] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.473649] Spectre V2 : Mitigation: Retpolines gitlab # [ 0.474648] RETBleed: Mitigation: untrained return thunk gitlab # [ 0.475648] Speculative Return Stack Overflow: Mitigation: SMT disabled gitlab # [ 0.477647] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.478650] Spectre V2 : Spectre v2 / SpectreRSB: Filling RSB on context switch and VMEXIT gitlab # [ 0.479647] Spectre V2 : Enabling Speculation Barrier for firmware calls gitlab # [ 0.480647] active return thunk: retbleed_return_thunk gitlab # [ 0.482652] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.483673] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.485647] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.487648] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.488650] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.489648] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'compacted' format. gitlab # [ 0.510451] Freeing SMP alternatives memory: 44K gitlab # [ 0.511650] pid_max: default: 32768 minimum: 301 gitlab # [ 0.512737] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.513730] landlock: Up and running. gitlab # [ 0.514647] Yama: becoming mindful. gitlab # [ 0.515821] LSM support for eBPF active gitlab # [ 0.516871] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.518824] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.522440] smpboot: CPU0: AMD Ryzen 9 3900 12-Core Processor (family: 0x17, model: 0x71, stepping: 0x0) gitlab # [ 0.523251] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.523654] ... version: 0 gitlab # [ 0.524651] ... bit width: 48 gitlab # [ 0.525664] ... generic counters: 6 gitlab # [ 0.526649] ... generic bitmap: 000000000000003f gitlab # [ 0.527649] ... fixed-purpose counters: 0 gitlab # [ 0.528649] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.529650] ... value mask: 0000ffffffffffff gitlab # [ 0.530650] ... max period: 00007fffffffffff gitlab # [ 0.531649] ... global_ctrl mask: 000000000000003f gitlab # [ 0.532740] signal: max sigframe size: 1776 gitlab # [ 0.533770] rcu: Hierarchical SRCU implementation. gitlab # [ 0.534651] rcu: Max phase no-delay instances is 400. gitlab # [ 0.535813] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.540447] smp: Bringing up secondary CPUs ... gitlab # [ 0.541787] smpboot: x86: Booting SMP configuration: gitlab # [ 0.542656] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.544684] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.546660] smpboot: Total of 4 processors activated (24799.98 BogoMIPS) gitlab # [ 0.548020] Memory: 6013776K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 258088K reserved, 0K cma-reserved) gitlab # [ 0.549193] devtmpfs: initialized gitlab # [ 0.550784] x86/mm: Memory block size: 128MB gitlab # [ 0.553679] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.554745] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.555828] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.557031] PM: RTC time: 08:45:24, date: 2026-09-01 gitlab # [ 0.560452] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.562945] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.563748] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.565761] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.566676] audit: initializing netlink subsys (disabled) gitlab # [ 0.567727] audit: type=2000 audit(1788252324.436:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.567942] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.568652] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.569649] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.570649] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.571649] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.572694] cpuidle: using governor menu gitlab # [ 0.575723] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.576958] PCI: Using configuration type 1 for base access gitlab # [ 0.577649] PCI: Using configuration type 1 for extended access gitlab # [ 0.578757] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.585973] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.586649] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.587649] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.588654] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.591730] ACPI: Added _OSI(Module Device) gitlab # [ 0.592648] ACPI: Added _OSI(Processor Device) gitlab # [ 0.593648] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.595956] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.598840] ACPI: Interpreter enabled gitlab # [ 0.600504] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.601595] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.602717] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.603651] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.604817] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.609345] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.610656] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.612001] acpiphp: Slot [3] registered gitlab # [ 0.612687] acpiphp: Slot [4] registered gitlab # [ 0.613710] acpiphp: Slot [5] registered gitlab # [ 0.614687] acpiphp: Slot [6] registered gitlab # [ 0.615689] acpiphp: Slot [7] registered gitlab # [ 0.616714] acpiphp: Slot [8] registered gitlab # [ 0.617684] acpiphp: Slot [9] registered gitlab # [ 0.618659] acpiphp: Slot [10] registered gitlab # [ 0.619639] acpiphp: Slot [11] registered gitlab # [ 0.620358] acpiphp: Slot [12] registered gitlab # [ 0.621458] acpiphp: Slot [13] registered gitlab # [ 0.622482] acpiphp: Slot [14] registered gitlab # [ 0.623489] acpiphp: Slot [15] registered gitlab # [ 0.624441] acpiphp: Slot [16] registered gitlab # [ 0.625430] acpiphp: Slot [17] registered gitlab # [ 0.626455] acpiphp: Slot [18] registered gitlab # [ 0.627557] acpiphp: Slot [19] registered gitlab # [ 0.628454] acpiphp: Slot [20] registered gitlab # [ 0.629510] acpiphp: Slot [21] registered gitlab # [ 0.630458] acpiphp: Slot [22] registered gitlab # [ 0.632464] acpiphp: Slot [23] registered gitlab # [ 0.633432] acpiphp: Slot [24] registered gitlab # [ 0.634443] acpiphp: Slot [25] registered gitlab # [ 0.635431] acpiphp: Slot [26] registered gitlab # [ 0.636436] acpiphp: Slot [27] registered gitlab # [ 0.637431] acpiphp: Slot [28] registered gitlab # [ 0.638445] acpiphp: Slot [29] registered gitlab # [ 0.639437] acpiphp: Slot [30] registered gitlab # [ 0.640438] acpiphp: Slot [31] registered gitlab # [ 0.641428] PCI host bridge to bus 0000:00 gitlab # [ 0.642429] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.643650] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.645650] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.647650] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.648649] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.650652] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.653125] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.655606] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.659728] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.663654] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.665720] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.667654] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.669653] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.671657] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.673074] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.677508] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.680864] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.684340] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.686674] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.689214] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.700717] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.702672] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.703691] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.705868] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.708639] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.714672] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 0.715664] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 0.716700] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 0.718667] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 0.722474] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.726642] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 0.732671] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 0.733764] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 0.737336] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.741291] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 0.742664] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 0.744706] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 0.747219] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.751663] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 0.752685] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 0.754700] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 0.758947] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.764661] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 0.765661] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 0.766703] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 0.770179] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.775681] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 0.776667] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 0.778697] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 0.782205] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.786671] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 0.787664] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 0.789700] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 0.791662] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 0.794288] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.797675] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 0.799731] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 0.803266] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.807670] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 0.808662] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 0.810694] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 0.814231] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.817474] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 0.818636] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 0.820736] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 0.829860] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 0.831824] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 0.833817] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 0.835817] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 0.837802] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 0.840950] iommu: Default domain type: Translated gitlab # [ 0.842651] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 0.844988] ACPI: bus type USB registered gitlab # [ 0.845739] usbcore: registered new interface driver usbfs gitlab # [ 0.847678] usbcore: registered new interface driver hub gitlab # [ 0.848673] usbcore: registered new device driver usb gitlab # [ 0.851735] NetLabel: Initializing gitlab # [ 0.852649] NetLabel: domain hash size = 128 gitlab # [ 0.853649] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 0.855712] NetLabel: unlabeled traffic allowed by default gitlab # [ 0.857674] PCI: Using ACPI for IRQ routing gitlab # [ 0.859801] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 0.860644] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 0.860644] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 0.864659] vgaarb: loaded gitlab # [ 0.865766] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 0.867649] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 0.874763] clocksource: Switched to clocksource kvm-clock gitlab # [ 0.876673] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.877670] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.879495] pnp: PnP ACPI init gitlab # [ 0.880699] pnp: PnP ACPI: found 6 devices gitlab # [ 0.888838] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 0.890900] clocksource: Switched to clocksource acpi_pm gitlab # [ 0.892458] NET: Registered PF_INET protocol family gitlab # [ 0.894839] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.911728] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.913903] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.915826] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.920058] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.921958] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.923764] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.926384] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.928449] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.930555] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.932185] NET: Registered PF_XDP protocol family gitlab # [ 0.933598] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 0.935165] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 0.936669] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 0.938291] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.939970] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.941899] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 0.943390] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 0.945813] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 0.948667] PCI: CLS 0 bytes, default 64 gitlab # [ 0.949841] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 0.950030] Trying to unpack rootfs image as initramfs... gitlab # [ 0.950186] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB) gitlab # [ 0.950287] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x2caf45e6a57, max_idle_ns: 440795365367 ns gitlab # [ 0.974956] Initialise system trusted keyrings gitlab # [ 0.976564] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 0.991409] Key type asymmetric registered gitlab # [ 0.992508] Asymmetric key parser 'x509' registered gitlab # [ 0.993892] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 0.996079] io scheduler mq-deadline registered gitlab # [ 0.997276] io scheduler kyber registered gitlab # [ 1.001547] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 1.003480] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 1.006404] Linux agpgart interface v0.103 gitlab # [ 1.007517] ACPI: bus type drm_connector registered gitlab # [ 1.010771] usbcore: registered new interface driver usbserial_generic gitlab # [ 1.012404] usbserial: USB Serial support registered for generic gitlab # [ 1.014089] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 1.016257] drop_monitor: Initializing network drop monitor service gitlab # [ 1.017933] NET: Registered PF_INET6 protocol family gitlab # [ 1.020418] Segment Routing with IPv6 gitlab # [ 1.021490] In-situ OAM (IOAM) with IPv6 gitlab # [ 1.024616] IPI shorthand broadcast: enabled gitlab # [ 1.028068] sched_clock: Marking stable (820021583, 207125451)->(1074040117, -46893083) gitlab # [ 1.030504] registered taskstats version 1 gitlab # [ 1.031780] Loading compiled-in X.509 certificates gitlab # [ 1.038478] Demotion targets for Node 0: null gitlab # [ 1.039968] Key type .fscrypt registered gitlab # [ 1.041097] Key type fscrypt-provisioning registered gitlab # [ 1.042649] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 1.044184] ima: Allocated hash algorithm: sha1 gitlab # [ 1.045533] ima: No architecture policies found gitlab # [ 1.047288] PM: Magic number: 10:176:775 gitlab # [ 1.049207] RAS: Correctable Errors collector initialized. gitlab # [ 1.054101] clk: Disabling unused clocks gitlab # [ 1.055103] PM: genpd: Disabling unused power domains gitlab # [ 1.263653] Freeing initrd memory: 29392K gitlab # [ 1.268588] Freeing unused decrypted memory: 2028K gitlab # [ 1.271409] Freeing unused kernel image (initmem) memory: 3640K gitlab # [ 1.272872] Write protecting the kernel read-only data: 32768k gitlab # [ 1.275165] Freeing unused kernel image (text/rodata gap) memory: 1220K gitlab # [ 1.277168] Freeing unused kernel image (rodata/data gap) memory: 760K gitlab # [ 1.310073] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.311650] Run /init as init process gitlab # [ 1.323297] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.345551] fuse: init (API version 7.45) gitlab # [ 1.353004] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 1.359897] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 1.363249] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 1.399311] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 1.736028] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.744188] systemd[1]: Detected virtualization kvm. gitlab # [ 1.745687] systemd[1]: Detected architecture x86-64. gitlab # [ 1.747192] systemd[1]: Running in initrd. gitlab # [ 1.748811] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.751000] systemd[1]: Hostname set to . gitlab # [ 1.847861] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 1.884405] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 1.900179] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 1.902240] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 1.904649] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 1.906600] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 1.908472] systemd[1]: Reached target Path Units. gitlab # [ 1.909960] systemd[1]: Reached target Slice Units. gitlab # [ 1.911571] systemd[1]: Reached target Swaps. gitlab # [ 1.912973] systemd[1]: Reached target Timer Units. gitlab # [ 1.914765] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 1.916850] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 1.918823] systemd[1]: Listening on Journal Sockets. gitlab # [ 1.920603] systemd[1]: Listening on udev Control Socket. gitlab # [ 1.922396] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 1.924036] systemd[1]: Reached target Socket Units. gitlab # [ 1.926835] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 1.929881] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 1.932723] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 1.936295] systemd[1]: Starting Journal Service... gitlab # [ 1.939218] systemd[1]: Starting Load Kernel Modules... gitlab # [ 1.940552] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 1.945220] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 1.947855] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 1.951004] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 1.953932] systemd-journald[92]: Collecting audit messages is disabled. gitlab # [ 1.972221] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 1.973077] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 1.974323] netfs: FS-Cache loaded gitlab # [ 1.978937] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 1.982186] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 1.986233] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 1.986938] 9pnet: Installing 9P2000 support gitlab # [ 1.998595] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.002269] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.006796] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.010805] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.014202] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.017702] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.020190] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 2.024497] systemd[1]: Started Journal Service. gitlab # [ 1.816848] systemd-modules-load[93]: Using 4 probe threads gitlab # [ 1.819810] systemd-modules-load[93]: Inserted module 'virtio_balloon' gitlab # [ 1.821499] systemd-modules-load[93]: Inserted module 'dm_mod' gitlab # [ 1.823044] systemd-modules-load[93]: Inserted module 'virtio_gpu' gitlab # [ 1.827801] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 1.829425] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 1.830984] systemd[1]: Reached target Local File Systems. gitlab # [ 1.832245] systemd[1]: Starting Create System Files and Directories... gitlab # [ 1.833674] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 1.849697] systemd[1]: Finished Create System Files and Directories. gitlab # [ 1.857440] systemd-udevd[112]: Using default interface naming scheme 'v261'. gitlab # [ 1.859158] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 1.868454] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 1.869984] systemd[1]: Reached target System Initialization. gitlab # [ 1.871124] systemd[1]: Reached target Basic System. gitlab # [ 2.195818] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 2.198187] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 2.199909] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 2.203550] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 2.205594] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 2.207533] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 2.209277] usb usb1: Product: UHCI Host Controller gitlab # [ 2.210364] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd gitlab # [ 2.210457] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 2.210512] SCSI subsystem initialized gitlab # [ 2.210674] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 2.214879] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 2.216915] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 2.219767] hub 1-0:1.0: USB hub found gitlab # [ 2.220777] hub 1-0:1.0: 2 ports detected gitlab # [ 2.247757] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 2.254596] scsi host0: ata_piix gitlab # [ 2.256005] scsi host1: ata_piix gitlab # [ 2.257658] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 2.257808] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 2.262488] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.264017] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 2.270471] virtio_blk virtio5: [vdb] 10454904 512-byte logical blocks (5.35 GB/4.98 GiB) gitlab # [ 2.063770] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.067635] (udev-worker)[124]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.072487] (udev-worker)[125]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.075546] (udev-worker)[125]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.080620] systemd-vconsole-setup[150]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.083537] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.422600] ata2: found unknown device (class 0) gitlab # [ 2.424284] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 2.426758] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 2.438167] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 2.459546] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # [ 2.263267] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.266280] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.267704] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.268975] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.489054] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 2.291166] systemd-fsck[158]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.294536] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.517640] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 2.605852] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 2.607551] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 2.608949] usb 1-1: Product: QEMU USB Tablet gitlab # [ 2.610006] usb 1-1: Manufacturer: QEMU gitlab # [ 2.610980] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 2.627831] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 2.636310] usbcore: registered new interface driver usbhid gitlab # [ 2.638004] usbhid: USB HID core driver gitlab # [ 2.642624] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 2.645161] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 2.743856] systemd[1]: Mounting /sysroot... gitlab # [ 3.067172] EXT4-fs (vda): mounted filesystem 66006670-0cbd-4e4d-afaf-d30e1d531023 r/w with ordered data mode. Quota mode: none. gitlab # [ 2.863606] systemd[1]: Mounted /sysroot. gitlab # [ 2.864569] systemd[1]: Reached target Initrd Root File System. gitlab # [ 2.865942] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 2.867046] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 2.868428] systemd[1]: Mounting /sysroot/run... gitlab # [ 2.878365] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 2.880609] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 2.884114] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.094634] 9p: Installing v9fs 9p2000 file system support gitlab # [ 3.095378] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 2.889513] systemd-sysroot-fstab-check[179]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 2.892119] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 2.893175] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 2.894226] systemd[1]: Mounted /sysroot/run. gitlab # [ 2.895177] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 2.897149] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 2.899539] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 2.900745] systemd[1]: Reload requested from client PID 179 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 2.902798] systemd[1]: Reloading... gitlab # [ 2.963847] systemd[1]: Reloading finished in 63 ms. gitlab # [ 2.992829] systemd-sysroot-fstab-check[179]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.003068] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.005352] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.007841] systemd-sysroot-fstab-check[179]: Requesting swap.target/start/replace... gitlab # [ 3.015764] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.018617] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.031771] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.033501] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.035751] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.037602] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.747623] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 3.778616] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 3.779782] systemd[1]: Reached target Initrd File Systems. gitlab # [ 3.781038] systemd[1]: Starting Find NixOS closure... gitlab # [ 3.782320] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 3.797453] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 3.799102] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 3.801782] systemd[1]: Finished Find NixOS closure. gitlab # [ 3.802994] systemd[1]: Reached target Initrd Default Target. gitlab # [ 3.804451] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 3.825983] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 3.827397] systemd[1]: Stopped target Basic System. gitlab # [ 3.828555] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 3.829709] systemd[1]: Stopped target Path Units. gitlab # [ 3.830819] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 3.832497] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 3.834241] systemd[1]: Stopped target Slice Units. gitlab # [ 3.836210] systemd[1]: Stopped target Socket Units. gitlab # [ 3.837405] systemd[1]: Stopped target System Initialization. gitlab # [ 3.839233] systemd[1]: Stopped target Swaps. gitlab # [ 3.840450] systemd[1]: Stopped target Timer Units. gitlab # [ 3.841757] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 3.843102] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 3.844360] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 3.845940] systemd[1]: Stopped Find NixOS closure. gitlab # [ 3.847052] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 3.849381] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.850615] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 3.851954] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 3.853094] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 3.854634] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 3.855796] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 3.857367] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 3.858966] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 3.860323] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 3.861515] systemd[1]: Stopped target Local File Systems. gitlab # [ 3.862653] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 3.864366] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 3.865836] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 3.866996] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 3.868506] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 3.869882] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 3.870888] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 3.872287] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 3.873774] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.875243] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.876415] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 3.877658] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 3.879083] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 3.880485] systemd[1]: Closed udev Control Socket. gitlab # [ 3.881749] systemd[1]: Starting Cleanup udev Database... gitlab # [ 3.882898] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 3.884577] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 3.886095] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 3.888159] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 3.889798] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 3.891315] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 3.892682] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 3.894248] systemd[1]: Finished Cleanup udev Database. gitlab # [ 3.895388] systemd[1]: Reached target Switch Root. gitlab # [ 3.896530] systemd[1]: Starting NixOS Activation... gitlab # [ 3.925864] initrd-nixos-activation-start[226]: booting system configuration /nix/store/zifp21wm254bs0qi6zlmc55z81l1bgph-nixos-system-gitlab-test gitlab # [ 3.942880] initrd-nixos-activation-start[226]: running activation script... gitlab # [ 4.116420] initrd-nixos-activation-start[249]: setting up /etc... gitlab # [ 4.195691] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.197497] systemd[1]: Finished NixOS Activation. gitlab # [ 4.199640] systemd[1]: Starting Switch Root... gitlab # [ 4.226281] systemd[1]: Switching root. gitlab # [ 4.475544] systemd-journald[92]: Received SIGTERM from PID 1 (systemd). gitlab # [ 4.540980] NET: Registered PF_VSOCK protocol family gitlab # [ 4.898716] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 4.904825] systemd[1]: Detected virtualization kvm. gitlab # [ 4.905969] systemd[1]: Detected architecture x86-64. gitlab # [ 4.907214] systemd[1]: Detected first boot. gitlab # [ 4.908602] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.011408] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.061033] systemd[1]: Applying preset policy. gitlab # [ 5.142183] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 5.265362] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 5.267539] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 5.270625] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 5.273657] systemd[1]: Created slice Slice /system/getty. gitlab # [ 5.275639] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 5.277750] systemd[1]: Created slice User and Session Slice. gitlab # [ 5.279262] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 5.281269] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 5.283129] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 5.284314] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 5.285580] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 5.287001] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 5.288292] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 5.289773] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 5.291155] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 5.292457] systemd[1]: Reached target Path Units. gitlab # [ 5.293500] systemd[1]: Reached target Remote File Systems. gitlab # [ 5.294679] systemd[1]: Reached target Slice Units. gitlab # [ 5.295732] systemd[1]: Reached target Swaps. gitlab # [ 5.297778] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 5.300835] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 5.303083] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 5.305667] systemd[1]: Listening on Factory Reset Management. gitlab # [ 5.307200] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 5.310522] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 5.312563] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 5.315477] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 5.317286] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 5.319165] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.321285] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 5.325001] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 5.326701] systemd[1]: Listening on udev Control Socket. gitlab # [ 5.328191] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 5.331344] systemd[1]: Mounting Huge Pages File System... gitlab # [ 5.333972] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 5.339249] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 5.342541] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 5.346338] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 5.347828] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 5.351404] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 5.352654] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 5.354568] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 5.356641] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 5.359646] systemd[1]: Mounting FUSE Control File System... gitlab # [ 5.360864] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 5.365223] systemd[1]: Starting Journal Service... gitlab # [ 5.367285] systemd[1]: Starting Load Kernel Modules... gitlab # [ 5.369758] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 5.378770] systemd-journald[319]: Collecting audit messages is enabled. gitlab # [ 5.378842] loop: module loaded gitlab # [ 5.379185] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 5.382163] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.387599] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 5.391371] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 5.392724] systemd[1]: Mounted Huge Pages File System. gitlab # [ 5.393979] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 5.395378] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 5.396626] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 5.402182] EXT4-fs (vda): re-mounted 66006670-0cbd-4e4d-afaf-d30e1d531023. gitlab # [ 5.402841] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 5.198253] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 5.407225] systemd[1]: Started Journal Service. gitlab # [ 5.200558] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 5.203168] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 5.205583] systemd-modules-load[320]: Using 3 probe threads gitlab # [ 5.207741] systemd-modules-load[320]: Inserted module 'loop' gitlab # [ 5.209531] systemd-modules-load[320]: Inserted module 'tls' gitlab # [ 5.211123] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 5.212951] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 5.214539] systemd[1]: Mounted FUSE Control File System. gitlab # [ 5.216181] systemd[1]: Finished Load Kernel Modules. gitlab # [ 5.217567] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 5.219360] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 5.223324] systemd-oomd[322]: No swap; memory pressure usage will be degraded gitlab # [ 5.227467] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 5.438244] systemd-journald[319]: Received client request to flush runtime journal. gitlab # [ 5.281424] systemd[1]: Starting Firewall... gitlab # [ 5.282774] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 5.284384] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 5.287113] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 5.289433] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 5.290797] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 5.292306] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.294846] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 5.296990] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 5.298864] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 5.300409] systemd[1]: Reached target First Boot Complete. gitlab # [ 5.301980] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 5.303464] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 5.305505] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 5.307334] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 5.309202] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 5.311226] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 5.313225] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 5.315201] systemd-udevd[354]: Using default interface naming scheme 'v261'. gitlab # [ 5.317118] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 5.329710] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 5.420922] systemd[1]: Finished Firewall. gitlab # [ 5.427313] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 5.456877] (udev-worker)[452]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 5.464713] (udev-worker)[461]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 5.467972] (udev-worker)[461]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 5.469891] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 5.481113] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 5.483049] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 5.484583] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 5.486671] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 5.489936] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 5.492662] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.494475] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.744252] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 5.751180] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 5.751672] ACPI: button: Power Button [PWRF] gitlab # [ 5.752824] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 5.760850] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 5.762657] rtc_cmos 00:05: registered as rtc0 gitlab # [ 5.763713] rtc_cmos 00:05: setting system clock to 2026-09-01T08:45:29 UTC (1788252329) gitlab # [ 5.765804] systemd-journald[319]: Time jumped backwards, rotating. gitlab # [ 5.766490] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 5.769213] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 5.774982] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 5.778529] Console: switching to colour dummy device 80x25 gitlab # [ 5.782039] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 5.782041] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 5.784338] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 5.785745] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 5.788180] FDC 0 is a S82078B gitlab # [ 5.803851] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 5.811683] Console: switching to colour frame buffer device 160x50 gitlab # [ 5.815572] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 5.816201] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 5.817960] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 5.824962] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input6 gitlab # [ 5.829173] mousedev: PS/2 mouse device common for all mice gitlab # [ 5.830539] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 gitlab # [ 5.700815] systemd[1]: Starting Virtual Console Setup... gitlab # [ 5.731554] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 5.733670] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 5.735568] systemd[1]: Starting Virtual Console Setup... gitlab # [ 5.947140] ppdev: user-space parallel port driver gitlab # [ 5.970113] kvm_amd: TSC scaling supported gitlab # [ 5.970751] kvm_amd: Nested Virtualization enabled gitlab # [ 5.971546] kvm_amd: Nested Paging enabled gitlab # [ 5.972226] kvm_amd: LBR virtualization supported gitlab # [ 5.972898] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 5.973692] kvm_amd: Virtual GIF supported gitlab # [ 5.770280] systemd-vconsole-setup[491]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 5.772573] systemd[1]: Finished Virtual Console Setup. gitlab # [ 6.019194] EDAC MC: Ver: 3.0.0 gitlab # [ 6.059853] systemd[1]: Mounting /run/wrappers... gitlab # [ 6.076086] systemd[1]: Mounted /run/wrappers. gitlab # [ 6.077390] systemd[1]: Reached target Local File Systems. gitlab # [ 6.078727] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 6.080563] systemd[1]: Starting register-nix-paths.service... gitlab # [ 6.082298] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 6.083732] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.086428] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 6.088448] systemd[1]: Starting Create System Files and Directories... gitlab # [ 6.104855] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 6.106690] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 6.109047] systemd[1]: Finished Create System Files and Directories. gitlab # [ 6.112278] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 6.113988] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 6.128509] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 6.139404] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 6.140909] systemd[1]: Starting Update is Completed... gitlab # [ 6.160641] systemd[1]: Finished Update is Completed. gitlab # [ 6.179407] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 6.181611] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 6.436813] systemd[1]: Finished register-nix-paths.service. gitlab # [ 6.438262] systemd[1]: Reached target System Initialization. gitlab # [ 6.439642] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 6.441167] systemd[1]: Started logrotate.timer. gitlab # [ 6.442166] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 6.443377] systemd[1]: Reached target Timer Units. gitlab # [ 6.444340] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 6.445539] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 6.446520] systemd[1]: Starting nixbot CI service socket... gitlab # [ 6.447606] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 6.449118] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 6.450226] systemd[1]: Reached target Socket Units. gitlab # [ 6.451288] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 6.469675] dbus-broker-launch[574]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 6.471763] dbus-broker-launch[574]: NSS returned no entry for 'systemd-timesync' gitlab # [ 6.473292] dbus-broker-launch[574]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 6.476564] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 6.484844] dbus-broker-launch[574]: Ready gitlab # [ 6.486714] systemd[1]: Reached target Basic System. gitlab # [ 6.487910] systemd[1]: Started backdoor.service. gitlab # [ 6.489247] systemd[1]: Starting gitlab-config.service... gitlab # [ 6.491470] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 6.506239] systemd[1]: Starting Logrotate configuration check... gitlab # [ 6.509231] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 6.512631] systemd[1]: Starting Post-Boot Actions... gitlab # [ 6.515044] logrotate[579]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # connecting to host... gitlab # [ 6.524280] logrotate[579]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf gitlab # [ 6.525283] logrotate[579]: note: 'monthly' overrides previously specified 'weekly' gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 6.525990] logrotate[579]: Reading state from file: /var/lib/logrotate.status gitlab # [ 6.526834] logrotate[579]: state file /var/lib/logrotate.status does not exist gitlab: connected to guest root shell gitlab # [ 6.530118] logrotate[579]: Allocating hash table for state file, size 64 entries gitlab: (connecting took 7.33 seconds) gitlab # [ 6.530961] logrotate[579]: Handling 4 logs gitlab: (finished: waiting for the VM to finish booting, in 7.33 seconds) gitlab # [ 6.532233] logrotate[579]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 6.536920] logrotate[579]: considering log /var/log/btmp gitlab # [ 6.537618] logrotate[579]: Creating new state gitlab # [ 6.538606] logrotate[579]: Now: 2026-09-01 08:45 gitlab # [ 6.539348] logrotate[579]: Last rotated at 2026-09-01 08:00 gitlab # [ 6.540091] logrotate[579]: log does not need rotating (log has already been rotated) gitlab # [ 6.540839] logrotate[579]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 6.541852] logrotate[579]: considering log /var/log/wtmp gitlab # [ 6.542581] logrotate[579]: Creating new state gitlab # [ 6.544126] logrotate[579]: Now: 2026-09-01 08:45 gitlab # [ 6.545391] logrotate[579]: Last rotated at 2026-09-01 08:00 gitlab # [ 6.546560] logrotate[579]: log does not need rotating (log has already been rotated) gitlab # [ 6.548450] logrotate[579]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 6.549183] logrotate[579]: switching euid from 0 to 165 and egid from 0 to 165 (pid 579) gitlab # [ 6.549921] logrotate[579]: considering log /var/gitlab/state/log/*.log gitlab # [ 6.551801] logrotate[579]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 6.552354] logrotate[579]: Creating new state gitlab # [ 6.553180] logrotate[579]: switching euid from 165 to 0 and egid from 165 to 0 (pid 579) gitlab # [ 6.553872] logrotate[579]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 6.555090] logrotate[579]: switching euid from 0 to 60 and egid from 0 to 60 (pid 579) gitlab # [ 6.555911] logrotate[579]: considering log /var/log/nginx/*.log gitlab # [ 6.557368] logrotate[579]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 6.558316] logrotate[579]: Creating new state gitlab # [ 6.558890] logrotate[579]: not running postrotate script, since no logs were rotated gitlab # [ 6.561308] logrotate[579]: switching euid from 60 to 0 and egid from 60 to 0 (pid 579) gitlab # [ 6.562474] systemd[1]: Started Reset console on configuration changes. gitlab # [ 6.563612] systemd[1]: Starting resolvconf update... gitlab # [ 6.564778] nsncd[580]: Sep 01 08:45:30.487 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 6.565970] systemd[1]: Finished Logrotate configuration check. gitlab # [ 6.567105] systemd[1]: Finished Post-Boot Actions. gitlab # [ 6.568227] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 6.585342] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 6.587543] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 6.591915] systemd[1]: Starting User Login Management... gitlab # [ 6.594205] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 6.604428] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 6.605482] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 6.607271] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 6.609212] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 6.610627] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 6.612615] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 6.616798] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 6.617626] systemd[1]: nscd.service: Consumed 16ms CPU time over 104ms wall clock time, 7.3M memory peak, 5.5M read from disk. gitlab # [ 6.623913] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 6.629733] systemd[1]: Finished resolvconf update. gitlab # [ 6.631038] systemd[1]: Reached target Preparation for Network. gitlab # [ 6.636167] systemd-logind[649]: New seat seat0. gitlab # [ 6.638205] systemd[1]: Starting DHCP Client... gitlab # [ 6.640463] nsncd[677]: Sep 01 08:45:30.582 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 6.643733] systemd[1]: Starting Address configuration of eth1... gitlab # [ 6.653367] systemd-logind[649]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 6.654436] systemd-logind[649]: Watching system buttons on /dev/input/event4 (QEMU Virtio Keyboard) gitlab # [ 6.655086] systemd-logind[649]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 6.657757] systemd[1]: Starting Extra networking commands.... gitlab # [ 6.658773] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 6.659829] systemd[1]: Started User Login Management. gitlab # [ 6.661352] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 6.662447] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 6.663829] systemd[1]: Starting linger-users.service... gitlab # [ 6.674713] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 6.675722] systemd[1]: Finished linger-users.service. gitlab # [ 6.685717] network-addresses-eth1-start[696]: adding address 192.168.1.1/24... done gitlab # [ 6.689832] network-addresses-eth1-start[696]: adding address 2001:db8:1::1/64... done gitlab # [ 6.692111] systemd[1]: Finished Address configuration of eth1. gitlab # [ 6.717153] dhcpcd[700]: dhcpcd-10.3.2 starting gitlab # [ 6.718301] systemd[1]: Finished Extra networking commands.. gitlab # [ 6.719711] systemd[1]: Reached target Network. gitlab # [ 6.720827] dhcpcd[761]: dev: loaded udev gitlab # [ 6.721719] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 6.724101] systemd[1]: Starting Nginx Web Server... gitlab # [ 6.933516] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 6.934223] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 6.728087] systemd[1]: Starting PostgreSQL Server... gitlab # [ 6.729555] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 6.733267] systemd[1]: Starting Permit User Sessions... gitlab # [ 6.745917] systemd[1]: Finished Permit User Sessions. gitlab # [ 6.754266] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 6.759644] systemd[1]: Started Getty on tty1. gitlab # [ 6.762254] systemd[1]: Reached target Login Prompts. gitlab # [ 7.033455] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 7.037689] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 7.038779] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 7.040163] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 7.041565] cfg80211: failed to load regulatory.db gitlab # [ 6.846407] nginx-pre-start[786]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok gitlab # [ 6.847359] nginx-pre-start[786]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful gitlab # [ 6.851864] systemd[1]: Started Nginx Web Server. gitlab # [ 7.073514] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 6.868599] dhcpcd[761]: eth0: waiting for carrier gitlab # [ 6.869771] dhcpcd[761]: eth0: carrier acquired gitlab # [ 6.873927] dhcpcd[761]: DUID 00:01:00:01:32:29:4d:2a:52:54:00:12:34:56 gitlab # [ 6.875303] dhcpcd[761]: eth0: IAID 00:12:34:56 gitlab # [ 6.876588] dhcpcd[761]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 6.880367] redis-server[784]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 6.884815] redis-server[784]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 6.885881] redis-server[784]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=784, just started gitlab # [ 6.886939] redis-server[784]: Configuration loaded gitlab # [ 6.888041] redis[784]: monotonic clock: POSIX clock_gettime gitlab # [ 6.888711] redis[784]: Running mode=standalone, port=0. gitlab # [ 6.892061] redis[784]: Server initialized gitlab # [ 6.892928] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 6.893952] redis[784]: Ready to accept connections unix gitlab # [ 6.895081] redis[784]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 6.911240] postgresql-pre-start[795]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 6.912169] postgresql-pre-start[795]: This user must also own the server process. gitlab # [ 6.915180] postgresql-pre-start[795]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 6.916920] postgresql-pre-start[795]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 6.918703] postgresql-pre-start[795]: The default text search configuration will be set to "english". gitlab # [ 6.920459] postgresql-pre-start[795]: Data page checksums are enabled. gitlab # [ 6.921360] postgresql-pre-start[795]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 6.922069] postgresql-pre-start[795]: creating subdirectories ... ok gitlab # [ 6.922704] postgresql-pre-start[795]: selecting dynamic shared memory implementation ... posix gitlab # [ 6.942399] postgresql-pre-start[795]: selecting default "max_connections" ... 100 gitlab # [ 6.970941] postgresql-pre-start[795]: selecting default "shared_buffers" ... 128MB gitlab # [ 7.169385] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 7.201661] postgresql-pre-start[795]: selecting default time zone ... UTC gitlab # [ 7.203496] postgresql-pre-start[795]: creating configuration files ... ok gitlab # [ 7.349760] postgresql-pre-start[795]: running bootstrap script ... ok gitlab # [ 7.396853] gitlab-workhorse[822]: time="2026-09-01T08:45:31Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 7.397831] gitlab-workhorse[822]: time="2026-09-01T08:45:31Z" level=info msg="Using redis/go-redis" gitlab # [ 7.398706] gitlab-workhorse[822]: time="2026-09-01T08:45:31Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 7.866308] postgresql-pre-start[795]: performing post-bootstrap initialization ... ok gitlab # [ 8.059175] postgresql-pre-start[795]: syncing data to disk ... ok gitlab # [ 8.060152] postgresql-pre-start[795]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 8.060931] postgresql-pre-start[795]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 8.062052] postgresql-pre-start[795]: Success. You can now start the database server using: gitlab # [ 8.062922] postgresql-pre-start[795]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 8.125597] postgres[846]: [846] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 8.128799] postgres[846]: [846] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 8.130488] postgres[846]: [846] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 8.132122] postgres[846]: [846] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 8.133951] postgres[856]: [856] LOG: database system was shut down at 2026-09-01 08:45:31 GMT gitlab # [ 8.139089] postgres[846]: [846] LOG: database system is ready to accept connections gitlab # [ 8.141703] systemd[1]: Started PostgreSQL Server. gitlab # [ 8.142684] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 8.262866] postgresql-setup-start[867]: CREATE DATABASE gitlab # [ 8.278208] postgresql-setup-start[872]: CREATE ROLE gitlab # [ 8.287107] postgresql-setup-start[874]: ALTER DATABASE gitlab # [ 8.305551] postgresql-setup-start[879]: CREATE ROLE gitlab # [ 8.307901] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 8.308961] systemd[1]: Reached target PostgreSQL. gitlab # [ 8.309976] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 8.389152] gitlab-postgresql-start[892]: CREATE DATABASE gitlab # [ 8.412088] gitlab-postgresql-start[896]: CREATE EXTENSION gitlab # [ 8.421826] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[884]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 8.424404] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[884]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 8.453820] systemd[1]: Finished gitlab-config.service. gitlab # [ 8.455890] systemd[1]: Started gitaly.service. gitlab # [ 8.458352] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 8.473875] gitlab-postgresql-start[898]: CREATE EXTENSION gitlab # [ 8.477810] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 8.830143] dhcpcd[761]: eth0: soliciting a DHCP lease gitlab # [ 9.043403] NET: Registered PF_PACKET protocol family gitlab # [ 8.840665] dhcpcd[761]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 8.849354] dhcpcd[761]: eth0: probing address 10.0.2.15/24 gitlab # [ 8.955621] dhcpcd[761]: eth0: soliciting an IPv6 router gitlab # [ 8.956802] dhcpcd[761]: eth0: Router Advertisement from fe80::2 gitlab # [ 8.957939] dhcpcd[761]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 8.959175] dhcpcd[761]: eth0: adding route to fec0::/64 gitlab # [ 8.960301] dhcpcd[761]: eth0: adding default route via fe80::2 gitlab # [ 8.968774] gitaly[912]: time="2026-09-01T08:45:32.911Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=912 gitlab # [ 8.969698] gitaly[912]: time="2026-09-01T08:45:32.912Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=912 gitlab # [ 8.970653] gitaly[912]: time="2026-09-01T08:45:32.912Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=912 gitlab # [ 8.971261] gitaly[912]: time="2026-09-01T08:45:32.912Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=912 gitlab # [ 8.971835] gitaly[912]: time="2026-09-01T08:45:32.912Z" level=info msg="Starting Gitaly" pid=912 version= gitlab # [ 9.143176] gitaly[912]: time="2026-09-01T08:45:33.086Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=158 pid=912 gitlab # [ 9.144198] gitaly[912]: time="2026-09-01T08:45:33.086Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=912 gitlab # [ 9.144905] gitaly[912]: time="2026-09-01T08:45:33.086Z" level=info msg="finished initializing command factory" duration_ms=0 pid=912 gitlab # [ 9.145751] gitaly[912]: time="2026-09-01T08:45:33.086Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-912/git-exec-721049378.d/git pid=912 gitlab # [ 9.155097] gitaly[912]: time="2026-09-01T08:45:33.097Z" level=info msg="finished detecting git version" duration_ms=2 pid=912 gitlab # [ 9.155976] gitaly[912]: time="2026-09-01T08:45:33.097Z" level=info msg="using Git version" pid=912 version=2.53.g gitlab # [ 9.157054] gitaly[912]: time="2026-09-01T08:45:33.098Z" level=info msg="clearing disk cache object folder" pid=912 storage=default gitlab # [ 9.158055] gitaly[912]: time="2026-09-01T08:45:33.098Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache623045522 pid=912 storage=default gitlab # [ 9.158818] gitaly[912]: time="2026-09-01T08:45:33.098Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache623045522 pid=912 storage=default gitlab # [ 9.159905] gitaly[912]: time="2026-09-01T08:45:33.098Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=912 gitlab # [ 9.160983] gitaly[912]: time="2026-09-01T08:45:33.098Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=912 gitlab # [ 9.162186] gitaly[912]: time="2026-09-01T08:45:33.098Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache623045522 clear_duration_ms=0 pid=912 storage=default gitlab # [ 9.214212] gitaly[912]: time="2026-09-01T08:45:33.157Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=912 gitlab # [ 9.215400] gitaly[912]: time="2026-09-01T08:45:33.157Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=912 schema=unix gitlab # [ 9.216332] gitaly[912]: time="2026-09-01T08:45:33.157Z" level=info msg="finished tempdir cleaner walk" pid=912 storage=default time_ms=0 gitlab # [ 9.225717] gitaly[912]: time="2026-09-01T08:45:33.157Z" level=info msg="listening at address" address=/run/gitaly/gitaly-912/sock.d/intern pid=912 schema=unix gitlab # [ 9.228520] gitaly[912]: time="2026-09-01T08:45:33.157Z" level=info msg="counting repositories" pid=912 gitlab # [ 9.230485] gitaly[912]: time="2026-09-01T08:45:33.166Z" level=info msg="starting to count repositories" pid=912 storage_path=/var/gitlab/state/repositories gitlab # [ 9.233281] gitaly[912]: time="2026-09-01T08:45:33.166Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=912 storage_path=/var/gitlab/state/repositories gitlab # [ 9.236611] gitaly[912]: time="2026-09-01T08:45:33.166Z" level=info msg="completed counting all repositories" counting_duration=0 pid=912 gitlab # [ 9.237647] gitaly[912]: time="2026-09-01T08:45:33.167Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=912 gitlab # [ 9.238454] gitaly[912]: time="2026-09-01T08:45:33.168Z" level=info msg="Started Gitaly" duration_ms=241 pid=912 gitlab # [ 9.239267] gitaly[912]: time="2026-09-01T08:45:33.168Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=912 gitlab # [ 9.240056] gitaly[912]: time="2026-09-01T08:45:33.168Z" level=info msg="maintenance: daily scheduled" pid=912 scheduled="2026-09-01 12:00:00 +0000 UTC" gitlab # [ 9.505477] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[915]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 9.506457] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[915]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 9.521497] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[915]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.314747] gitaly[912]: time="2026-09-01T08:45:36.256Z" level=info msg="License database preloaded" duration_ms=3344 pid=912 gitlab # [ 13.533486] dhcpcd[761]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 13.534800] dhcpcd[761]: eth0: adding route to 10.0.2.0/24 gitlab # [ 13.535708] dhcpcd[761]: eth0: adding default route via 10.0.2.2 gitlab # [ 13.583123] systemd[1]: Started DHCP Client. gitlab # [ 13.584750] systemd[1]: Reached target Network is Online. gitlab # [ 42.960963] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[915]: W, [2026-09-01T08:46:06.903868 #915] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 45.098948] postgres[1341]: [1341] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 45.099793] postgres[1341]: [1341] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 45.100385] postgres[1341]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 45.101457] postgres[1341]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 45.102528] postgres[1341]: attidentity AS identity, gitlab # [ 45.103637] postgres[1341]: attgenerated as attgenerated gitlab # [ 45.104723] postgres[1341]: FROM pg_attribute a gitlab # [ 45.105855] postgres[1341]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 45.106975] postgres[1341]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 45.108058] postgres[1341]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 45.108902] postgres[1341]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 45.109983] postgres[1341]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 45.111141] postgres[1341]: ORDER BY a.attnum gitlab # [ 45.111958] postgres[1341]: gitlab # [ 47.461468] gitlab-workhorse[822]: time="2026-09-01T08:46:11Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 48.921268] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1383]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 48.922311] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1383]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 57.471352] gitlab-workhorse[822]: time="2026-09-01T08:46:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 65.854559] postgres[1467]: [1467] ERROR: relation "table_without_models" does not exist gitlab # [ 65.855622] postgres[1467]: [1467] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 67.481124] gitlab-workhorse[822]: time="2026-09-01T08:46:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 77.490726] gitlab-workhorse[822]: time="2026-09-01T08:46:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 79.397897] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 79.401150] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 79.402581] systemd[1]: Starting gitlab.service... gitlab # [ 79.856930] bundle[1661]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 79.858121] bundle[1661]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 79.859579] bundle[1661]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 83.453289] bundle[1661]: {"timestamp":"2026-09-01T08:46:47.372Z","pid":1661,"message":"* Enabling systemd notification integration"} gitlab # [ 83.454292] bundle[1661]: {"timestamp":"2026-09-01T08:46:47.396Z","pid":1661,"message":"Puma starting in cluster mode..."} gitlab # [ 83.454856] bundle[1661]: {"timestamp":"2026-09-01T08:46:47.396Z","pid":1661,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 83.455838] bundle[1661]: {"timestamp":"2026-09-01T08:46:47.396Z","pid":1661,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 83.456960] bundle[1661]: {"timestamp":"2026-09-01T08:46:47.396Z","pid":1661,"message":"* Min threads: 0"} gitlab # [ 83.457826] bundle[1661]: {"timestamp":"2026-09-01T08:46:47.396Z","pid":1661,"message":"* Max threads: 4"} gitlab # [ 83.458668] bundle[1661]: {"timestamp":"2026-09-01T08:46:47.396Z","pid":1661,"message":"* Environment: production"} gitlab # [ 83.459291] bundle[1661]: {"timestamp":"2026-09-01T08:46:47.396Z","pid":1661,"message":"* Master PID: 1661"} gitlab # [ 83.459908] bundle[1661]: {"timestamp":"2026-09-01T08:46:47.396Z","pid":1661,"message":"* Workers: 2"} gitlab # [ 83.460750] bundle[1661]: {"timestamp":"2026-09-01T08:46:47.396Z","pid":1661,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 83.461555] bundle[1661]: {"timestamp":"2026-09-01T08:46:47.397Z","pid":1661,"message":"* Preloading application"} gitlab # [ 83.509838] sidekiq-cluster[1660]: {"severity":"INFO","time":"2026-09-01T08:46:47.432Z","message":"Starting cluster with 1 processes"} gitlab # [ 87.500165] gitlab-workhorse[822]: time="2026-09-01T08:46:51Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 97.510745] gitlab-workhorse[822]: time="2026-09-01T08:47:01Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 107.511889] gitlab-workhorse[822]: time="2026-09-01T08:47:11Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 109.778729] bundle[1661]: W, [2026-09-01T08:47:13.721740 #1661] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 113.193459] sidekiq-cluster[1700]: 2026-09-01T08:47:17.136Z pid=1700 tid=wek INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1700, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>451788800} gitlab # [ 113.428427] sidekiq-cluster[1700]: W, [2026-09-01T08:47:17.371622 #1700] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 113.480705] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:17.423Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 115.742955] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.685Z","message":"GitLab reliable fetch activated!"} gitlab # [ 115.743935] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.685Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 115.781336] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.724Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 115.786788] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.730Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 115.792836] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.736Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 115.799071] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.742Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 115.803511] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.746Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 115.809356] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.752Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 115.815716] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.758Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 115.821431] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.764Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 115.826527] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.769Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 115.831451] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.774Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 115.836853] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.780Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 115.842720] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.785Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 115.848174] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.791Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 115.853981] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.797Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 115.859322] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.802Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 115.864516] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.807Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 115.871143] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.814Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 115.877378] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.820Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 115.883840] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.827Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 115.890328] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.833Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 115.897128] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.840Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 115.903359] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.846Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 115.910450] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.853Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 115.927682] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.870Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 115.943098] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.885Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 115.951982] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.894Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 115.965137] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.907Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 115.986374] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.927Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 115.996460] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.939Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 116.003449] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.946Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 116.009524] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.952Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 116.015698] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.958Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 116.020978] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.964Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 116.026331] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.969Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 116.031498] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.974Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 116.036474] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.979Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 116.041574] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.984Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 116.046883] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.990Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 116.052575] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:19.995Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 116.058112] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.001Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 116.064277] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.007Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 116.070434] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.013Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 116.076816] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.020Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 116.082653] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.025Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 116.085750] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.028Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 116.091822] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.035Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 116.097061] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.040Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 116.102647] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.045Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 116.108336] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.051Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 116.113756] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.057Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 116.119688] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.062Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 116.125800] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.069Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 116.156751] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.099Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 116.164614] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.107Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 116.170198] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.113Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 116.177746] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.120Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 116.182815] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.126Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 116.189598] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.132Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 116.194804] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.138Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 116.201186] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.144Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 116.206761] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.150Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 116.212984] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.156Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 116.219708] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.162Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 116.226309] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.169Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 116.233771] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.176Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 116.240635] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.183Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 116.247124] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.190Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 116.252943] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.196Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 116.259467] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.202Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 116.265205] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.208Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 116.270451] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.213Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 116.276262] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.219Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 116.281486] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.224Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 116.287615] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.230Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 116.293166] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.236Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 116.299277] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.242Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 116.305227] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.248Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 116.311949] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.255Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 116.318141] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.261Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 116.323547] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.266Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 116.329772] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.272Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 116.336264] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.279Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 116.342067] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.285Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 116.347736] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.291Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 116.353413] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.296Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 116.359048] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.302Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 116.364700] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.307Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 116.369802] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.313Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 116.375446] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.318Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 116.380778] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.323Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 116.386180] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.329Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 116.391971] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.335Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 116.398220] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.341Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 116.404390] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.347Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 116.410886] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.354Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 116.417839] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.361Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 116.424129] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.367Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 116.430685] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.373Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 116.443523] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.386Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 116.449857] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.393Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 116.456136] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.399Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 116.462227] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.405Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 116.468184] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.411Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 116.474163] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.417Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 116.481154] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.424Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 116.487441] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.430Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 116.493170] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.436Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 116.500347] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.443Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 116.506876] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:20.450Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 117.520951] gitlab-workhorse[822]: time="2026-09-01T08:47:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 127.523074] gitlab-workhorse[822]: time="2026-09-01T08:47:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 137.523841] gitlab-workhorse[822]: time="2026-09-01T08:47:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 140.520520] postgres[2238]: [2238] ERROR: relation "table_without_models" does not exist gitlab # [ 140.522034] postgres[2238]: [2238] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 142.158481] postgres[2249]: [2249] ERROR: relation "table_without_models" does not exist gitlab # [ 142.160269] postgres[2249]: [2249] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 143.902451] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:47.845Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 143.907169] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:47.848Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 143.908083] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:47.848Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 143.908745] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:47.848Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 143.909407] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:47.849Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 145.472507] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:49.415Z","message":"Cleaning working queues"} gitlab # [ 145.475480] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:49.418Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"10e6d0cd2d4485087cc3deb5","created_at":"2026-09-01T08:46:43.050Z","trace_propagation_headers":{"sentry-trace":"1e4926ea00b34049ae630666a2b89d8b-ba31642835a74cb6","baggage":"sentry-trace_id=1e4926ea00b34049ae630666a2b89d8b,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"fae02ff8f93f52a53bafdccfb1486a26","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-01T08:46:43.060Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-10e6d0cd2d4485087cc3deb5: start","job_status":"start","queue_duration_s":66.358403,"scheduling_latency_s":66.358444} gitlab # [ 145.623450] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:47:49.566Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"10e6d0cd2d4485087cc3deb5","created_at":"2026-09-01T08:46:43.050Z","trace_propagation_headers":{"sentry-trace":"1e4926ea00b34049ae630666a2b89d8b-ba31642835a74cb6","baggage":"sentry-trace_id=1e4926ea00b34049ae630666a2b89d8b,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"fae02ff8f93f52a53bafdccfb1486a26","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-01T08:46:43.060Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-10e6d0cd2d4485087cc3deb5: done: 0.147894 sec","job_status":"done","queue_duration_s":66.358403,"scheduling_latency_s":66.358444,"redis_calls":31,"redis_duration_s":0.006791,"redis_read_bytes":284,"redis_write_bytes":3897,"redis_feature_flag_calls":20,"redis_feature_flag_duration_s":0.004128,"redis_feature_flag_read_bytes":274,"redis_feature_flag_write_bytes":2650,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00169,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":16,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.073101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.072585482,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.147894,"completed_at":"2026-09-01T08:47:49.566Z","load_balancing_strategy":"primary","db_duration_s":0.035603,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 145.855269] bundle[1661]: {"timestamp":"2026-09-01T08:47:49.797Z","pid":1661,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 145.856108] bundle[1661]: {"timestamp":"2026-09-01T08:47:49.797Z","pid":1661,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 145.856746] bundle[1661]: {"timestamp":"2026-09-01T08:47:49.797Z","pid":1661,"message":"! #\u003cThread:0x00007ae69b9fb4e0 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 145.857685] bundle[1661]: {"timestamp":"2026-09-01T08:47:49.797Z","pid":1661,"message":"! #\u003cThread:0x00007ae69bc69f10 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 145.858315] bundle[1661]: {"timestamp":"2026-09-01T08:47:49.798Z","pid":1661,"message":"Use Ctrl-C to stop"} gitlab # [ 145.872858] bundle[1661]: {"timestamp":"2026-09-01T08:47:49.798Z","pid":1661,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 146.216036] bundle[1661]: {"timestamp":"2026-09-01T08:47:50.159Z","pid":1661,"message":"- Worker 0 (PID: 2321) booted in 0.32s, phase: 0"} gitlab # [ 146.230527] bundle[1661]: {"timestamp":"2026-09-01T08:47:50.173Z","pid":1661,"message":"- Worker 1 (PID: 2325) booted in 0.32s, phase: 0"} gitlab # [ 146.234476] systemd[1]: Started gitlab.service. gitlab # [ 146.235413] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 146.237113] systemd[1]: Starting setup-gitlab.service... gitlab # [ 146.252040] setup-gitlab-start[2349]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 148.623429] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:47:52 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 2354 gitlab # [ 148.625901] setup-gitlab-start[2349]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 148.626747] setup-gitlab-start[2349]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 148.638983] runuser[2372]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 149.091622] setup-gitlab-start[2373]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 149.092917] setup-gitlab-start[2373]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 149.094064] setup-gitlab-start[2373]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 149.793357] setup-gitlab-start[2373]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 149.794509] setup-gitlab-start[2373]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 149.795933] setup-gitlab-start[2373]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 158.755785] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:02.698Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9e37760f5b86f4648e6a41d0","created_at":"2026-09-01T08:48:02.680Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8b2fd1244fe2954771f3bf7573faa3d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:02.696Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9e37760f5b86f4648e6a41d0: start","job_status":"start","queue_duration_s":0.001666,"scheduling_latency_s":0.001694} gitlab # [ 158.787732] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:02.730Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"9e37760f5b86f4648e6a41d0","created_at":"2026-09-01T08:48:02.680Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8b2fd1244fe2954771f3bf7573faa3d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:02.696Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-9e37760f5b86f4648e6a41d0: done: 0.032116 sec","job_status":"done","queue_duration_s":0.001666,"scheduling_latency_s":0.001694,"redis_calls":12,"redis_duration_s":0.0074990000000000005,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000399,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005641,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001459,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000768254,"exclusive_lock_hold_duration_s":0.009239721999999999,"duration_s":0.032116,"completed_at":"2026-09-01T08:48:02.730Z","load_balancing_strategy":"primary","db_duration_s":0.002665,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.848118] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:02.791Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"3993ace8c2db1ee6a511c4a0","created_at":"2026-09-01T08:48:02.725Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a420494a038189d0d6eba431a1c7f061","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T08:48:02.790Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-3993ace8c2db1ee6a511c4a0: start","job_status":"start","queue_duration_s":0.001078,"scheduling_latency_s":0.001108} gitlab # [ 158.908199] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:02.851Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"3993ace8c2db1ee6a511c4a0","created_at":"2026-09-01T08:48:02.725Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a420494a038189d0d6eba431a1c7f061","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T08:48:02.790Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-3993ace8c2db1ee6a511c4a0: done: 0.060081 sec","job_status":"done","queue_duration_s":0.001078,"scheduling_latency_s":0.001108,"redis_calls":10,"redis_duration_s":0.008016,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001264,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00438,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002372,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000848711,"duration_s":0.060081,"completed_at":"2026-09-01T08:48:02.851Z","load_balancing_strategy":"primary","db_duration_s":0.00549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 158.938926] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:02.852Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0dd7ab52f87befd692d679fb","created_at":"2026-09-01T08:48:02.815Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fcbc1b6b86e1d4994d72ce7f8c6275a1","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:02.828Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0dd7ab52f87befd692d679fb: start","job_status":"start","queue_duration_s":0.024211,"scheduling_latency_s":0.024247} gitlab # [ 158.957941] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:02.900Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0dd7ab52f87befd692d679fb","created_at":"2026-09-01T08:48:02.815Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fcbc1b6b86e1d4994d72ce7f8c6275a1","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:02.828Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0dd7ab52f87befd692d679fb: done: 0.048251 sec","job_status":"done","queue_duration_s":0.024211,"scheduling_latency_s":0.024247,"redis_calls":13,"redis_duration_s":0.017808,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000513,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.015562,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001733,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.018818592000000002,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.048251,"completed_at":"2026-09-01T08:48:02.900Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00306,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 158.977365] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:02.904Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"944897a1dc85bf5c8ece2f57","created_at":"2026-09-01T08:48:02.843Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebbc930c235f28a9e1a4af8fdfcca76c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:02.877Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-944897a1dc85bf5c8ece2f57: start","job_status":"start","queue_duration_s":0.027083,"scheduling_latency_s":0.027109} gitlab # [ 159.043116] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:02.963Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"944897a1dc85bf5c8ece2f57","created_at":"2026-09-01T08:48:02.843Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ebbc930c235f28a9e1a4af8fdfcca76c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:02.877Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-944897a1dc85bf5c8ece2f57: done: 0.058863 sec","job_status":"done","queue_duration_s":0.027083,"scheduling_latency_s":0.027109,"redis_calls":10,"redis_duration_s":0.005785,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000849,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004393,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000543,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01553,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.002174019,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.058863,"completed_at":"2026-09-01T08:48:02.963Z","load_balancing_strategy":"primary","db_duration_s":0.019879,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 159.046335] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:02.968Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d999e4ff6a47c27586f1b1ed","created_at":"2026-09-01T08:48:02.892Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"818d2f0c61456021ed87f93754d40e3c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:02.907Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d999e4ff6a47c27586f1b1ed: start","job_status":"start","queue_duration_s":0.060979,"scheduling_latency_s":0.061011} gitlab # [ 160.094852] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.037Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d999e4ff6a47c27586f1b1ed","created_at":"2026-09-01T08:48:02.892Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"818d2f0c61456021ed87f93754d40e3c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:02.907Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d999e4ff6a47c27586f1b1ed: done: 1.069022 sec","job_status":"done","queue_duration_s":0.060979,"scheduling_latency_s":0.061011,"redis_calls":565,"redis_duration_s":0.702417,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000592,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004477,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.697348,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.39705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001489575,"duration_s":1.069022,"completed_at":"2026-09-01T08:48:04.037Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002096,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.097915] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.041Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"2ca9f5342ebc3c7c11ac3b5c","created_at":"2026-09-01T08:48:02.915Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9065ce3080f48e88c742cb7ee132eb9","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:02.928Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-2ca9f5342ebc3c7c11ac3b5c: start","job_status":"start","queue_duration_s":1.112097,"scheduling_latency_s":1.112122} gitlab # [ 160.206819] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.149Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"2ca9f5342ebc3c7c11ac3b5c","created_at":"2026-09-01T08:48:02.915Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9065ce3080f48e88c742cb7ee132eb9","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:02.928Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-2ca9f5342ebc3c7c11ac3b5c: done: 0.108681 sec","job_status":"done","queue_duration_s":1.112097,"scheduling_latency_s":1.112122,"redis_calls":16,"redis_duration_s":0.014542,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.006425,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005583,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002534,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.058,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.058,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023979,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000758756,"duration_s":0.108681,"completed_at":"2026-09-01T08:48:04.149Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.059778,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.268497] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.211Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"879d7b9d4b3d02c4473886c5","created_at":"2026-09-01T08:48:02.936Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40e188de7a467fad70c485b1b1aedcbe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:02.945Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-879d7b9d4b3d02c4473886c5: start","job_status":"start","queue_duration_s":1.26622,"scheduling_latency_s":1.266252} gitlab # [ 160.287686] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.229Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 160.303550] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.246Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"879d7b9d4b3d02c4473886c5","created_at":"2026-09-01T08:48:02.936Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"40e188de7a467fad70c485b1b1aedcbe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:02.945Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-879d7b9d4b3d02c4473886c5: done: 0.03499 sec","job_status":"done","queue_duration_s":1.26622,"scheduling_latency_s":1.266252,"redis_calls":10,"redis_duration_s":0.006056,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001889,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002868,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001299,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001403531,"duration_s":0.03499,"completed_at":"2026-09-01T08:48:04.246Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00225,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.360725] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.303Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"1296197f841b53136eee117a","created_at":"2026-09-01T08:48:02.953Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"204c0014316503a772c35580354d4737","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:02.965Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-1296197f841b53136eee117a: start","job_status":"start","queue_duration_s":1.338125,"scheduling_latency_s":1.338162} gitlab # [ 160.425752] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"1296197f841b53136eee117a","created_at":"2026-09-01T08:48:02.953Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"204c0014316503a772c35580354d4737","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:02.965Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-1296197f841b53136eee117a: done: 0.065001 sec","job_status":"done","queue_duration_s":1.338125,"scheduling_latency_s":1.338162,"redis_calls":12,"redis_duration_s":0.015647,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002734,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007481,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005432,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012487,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001452698,"exclusive_lock_hold_duration_s":0.007644268,"duration_s":0.065001,"completed_at":"2026-09-01T08:48:04.368Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00452,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.454929] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.374Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"5d8604228dc6be6cbcffb756","created_at":"2026-09-01T08:48:02.975Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c71a960ffd3b8dcc0ec1c8b65b66b1e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T08:48:02.993Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-5d8604228dc6be6cbcffb756: start","job_status":"start","queue_duration_s":1.380971,"scheduling_latency_s":1.380999} gitlab # [ 160.500598] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.398Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 160.505208] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.412Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"5d8604228dc6be6cbcffb756","created_at":"2026-09-01T08:48:02.975Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c71a960ffd3b8dcc0ec1c8b65b66b1e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-01T08:48:02.993Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-5d8604228dc6be6cbcffb756: done: 0.037636 sec","job_status":"done","queue_duration_s":1.380971,"scheduling_latency_s":1.380999,"redis_calls":10,"redis_duration_s":0.012013000000000001,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001443,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008214,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002356,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001847162,"duration_s":0.037636,"completed_at":"2026-09-01T08:48:04.412Z","load_balancing_strategy":"primary","db_duration_s":0.002572,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.506704] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.416Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7bd0e22a8283de0ca28f6dee","created_at":"2026-09-01T08:48:03.003Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc51dcf5916b6d0d6e1473d620114fb8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.010Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7bd0e22a8283de0ca28f6dee: start","job_status":"start","queue_duration_s":1.406021,"scheduling_latency_s":1.406048} gitlab # [ 160.587130] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.529Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7bd0e22a8283de0ca28f6dee","created_at":"2026-09-01T08:48:03.003Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc51dcf5916b6d0d6e1473d620114fb8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.010Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7bd0e22a8283de0ca28f6dee: done: 0.113211 sec","job_status":"done","queue_duration_s":1.406021,"scheduling_latency_s":1.406048,"redis_calls":12,"redis_duration_s":0.07895200000000001,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002073,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.075453,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001426,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.07132359,"duration_s":0.113211,"completed_at":"2026-09-01T08:48:04.529Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003703,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.648966] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.592Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"3be9a127059dab97275ed400","created_at":"2026-09-01T08:48:03.023Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"868a2b1c3e5831f047a2f01561387f61","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.029Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-3be9a127059dab97275ed400: start","job_status":"start","queue_duration_s":1.562173,"scheduling_latency_s":1.562202} gitlab # [ 160.687970] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.631Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 160.757113] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.700Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"3be9a127059dab97275ed400","created_at":"2026-09-01T08:48:03.023Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"868a2b1c3e5831f047a2f01561387f61","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.029Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-3be9a127059dab97275ed400: done: 0.108045 sec","job_status":"done","queue_duration_s":1.562173,"scheduling_latency_s":1.562202,"redis_calls":10,"redis_duration_s":0.010047,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001644,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00713,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001273,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.066461,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001771454,"duration_s":0.108045,"completed_at":"2026-09-01T08:48:04.699Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00119,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 160.812498] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.755Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c86651832efc7e1e1599a220","created_at":"2026-09-01T08:48:03.039Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd6dbd1c28df122ecfd7942aa6314300","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.049Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c86651832efc7e1e1599a220: start","job_status":"start","queue_duration_s":1.706261,"scheduling_latency_s":1.706296} gitlab # [ 160.978540] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.921Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c86651832efc7e1e1599a220","created_at":"2026-09-01T08:48:03.039Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd6dbd1c28df122ecfd7942aa6314300","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.049Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c86651832efc7e1e1599a220: done: 0.165875 sec","job_status":"done","queue_duration_s":1.706261,"scheduling_latency_s":1.706296,"redis_calls":14,"redis_duration_s":0.014064,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003819,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004718,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005527,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002351137,"exclusive_lock_hold_duration_s":0.108379976,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.165875,"completed_at":"2026-09-01T08:48:04.921Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044926,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.032794] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:04.975Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7c8beb708899edd17d9aa963","created_at":"2026-09-01T08:48:03.060Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e806075430c2d21f53695bcfe78b87f6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.072Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7c8beb708899edd17d9aa963: start","job_status":"start","queue_duration_s":1.903782,"scheduling_latency_s":1.903813} gitlab # [ 161.112419] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.054Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7c8beb708899edd17d9aa963","created_at":"2026-09-01T08:48:03.060Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e806075430c2d21f53695bcfe78b87f6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.072Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7c8beb708899edd17d9aa963: done: 0.078873 sec","job_status":"done","queue_duration_s":1.903782,"scheduling_latency_s":1.903813,"redis_calls":12,"redis_duration_s":0.011007000000000001,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001801,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006316,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00289,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000691708,"exclusive_lock_hold_duration_s":0.018545374,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.078873,"completed_at":"2026-09-01T08:48:05.054Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019016,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.174471] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.117Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"893fe8b15f559ea8e4f143fe","created_at":"2026-09-01T08:48:03.080Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"484b89a2aa4ce12c7345853662f7185e","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.088Z","job_size_bytes":3,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-893fe8b15f559ea8e4f143fe: start","job_status":"start","queue_duration_s":2.028986,"scheduling_latency_s":2.029055} gitlab # [ 161.228909] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.171Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"893fe8b15f559ea8e4f143fe","created_at":"2026-09-01T08:48:03.080Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"484b89a2aa4ce12c7345853662f7185e","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.088Z","job_size_bytes":3,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-893fe8b15f559ea8e4f143fe: done: 0.054472 sec","job_status":"done","queue_duration_s":2.028986,"scheduling_latency_s":2.029055,"redis_calls":10,"redis_duration_s":0.013748,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003411,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007249,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003088,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000662933,"duration_s":0.054472,"completed_at":"2026-09-01T08:48:05.171Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001997,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.258121] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.178Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"45586dc3fada28697fa0016b","created_at":"2026-09-01T08:48:03.095Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2c480b302f3b39d07d5d679379a8cdc","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.103Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-45586dc3fada28697fa0016b: start","job_status":"start","queue_duration_s":2.07515,"scheduling_latency_s":2.075169} gitlab # [ 161.320127] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.263Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"45586dc3fada28697fa0016b","created_at":"2026-09-01T08:48:03.095Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2c480b302f3b39d07d5d679379a8cdc","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.103Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-45586dc3fada28697fa0016b: done: 0.084117 sec","job_status":"done","queue_duration_s":2.07515,"scheduling_latency_s":2.075169,"redis_calls":12,"redis_duration_s":0.057083999999999996,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.044277,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001037,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008199,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003571,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000702603,"duration_s":0.084117,"completed_at":"2026-09-01T08:48:05.262Z","load_balancing_strategy":"primary","db_duration_s":0.002492,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.351967] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"deb3e99f53ae017e91fe433e","created_at":"2026-09-01T08:48:03.115Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba8f968efb9a57d4d25fa27d8627a027","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.124Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-deb3e99f53ae017e91fe433e: start","job_status":"start","queue_duration_s":2.14156,"scheduling_latency_s":2.141584} gitlab # [ 161.409266] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.352Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"deb3e99f53ae017e91fe433e","created_at":"2026-09-01T08:48:03.115Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba8f968efb9a57d4d25fa27d8627a027","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.124Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-deb3e99f53ae017e91fe433e: done: 0.086101 sec","job_status":"done","queue_duration_s":2.14156,"scheduling_latency_s":2.141584,"redis_calls":10,"redis_duration_s":0.0064410000000000005,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001842,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003226,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001373,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001699378,"duration_s":0.086101,"completed_at":"2026-09-01T08:48:05.352Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.051139,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.470532] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.413Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"7030828f471c1bd8c4005a7f","created_at":"2026-09-01T08:48:03.132Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47e46660c23bc762b9aaa48417a98f01","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.141Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-7030828f471c1bd8c4005a7f: start","job_status":"start","queue_duration_s":2.272108,"scheduling_latency_s":2.272145} gitlab # [ 161.523111] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"7030828f471c1bd8c4005a7f","created_at":"2026-09-01T08:48:03.132Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47e46660c23bc762b9aaa48417a98f01","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.141Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-7030828f471c1bd8c4005a7f: done: 0.052647 sec","job_status":"done","queue_duration_s":2.272108,"scheduling_latency_s":2.272145,"redis_calls":10,"redis_duration_s":0.009042,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001823,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00469,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002529,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00128955,"duration_s":0.052647,"completed_at":"2026-09-01T08:48:05.465Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001784,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.556868] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.467Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"76ab365f13701020d180b0f3","created_at":"2026-09-01T08:48:03.150Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a4affb642de580d46dfb27ac5bfa9ce","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.159Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-76ab365f13701020d180b0f3: start","job_status":"start","queue_duration_s":2.30892,"scheduling_latency_s":2.308945} gitlab # [ 161.779573] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.722Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"76ab365f13701020d180b0f3","created_at":"2026-09-01T08:48:03.150Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2a4affb642de580d46dfb27ac5bfa9ce","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.159Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-76ab365f13701020d180b0f3: done: 0.254417 sec","job_status":"done","queue_duration_s":2.30892,"scheduling_latency_s":2.308945,"redis_calls":14,"redis_duration_s":0.011233,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003228,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00415,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003855,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.06,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.06,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.120193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.002525181,"exclusive_lock_hold_duration_s":0.218630859,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.254417,"completed_at":"2026-09-01T08:48:05.722Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.119957,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.837370] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.756Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"3be54bfa73a1238087f0a1f3","created_at":"2026-09-01T08:48:03.169Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3244dd22ac647912e6a1ef987dec5475","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.177Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-3be54bfa73a1238087f0a1f3: start","job_status":"start","queue_duration_s":2.578399,"scheduling_latency_s":2.578429} gitlab # [ 161.899880] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.842Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"3be54bfa73a1238087f0a1f3","created_at":"2026-09-01T08:48:03.169Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3244dd22ac647912e6a1ef987dec5475","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.177Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-3be54bfa73a1238087f0a1f3: done: 0.086151 sec","job_status":"done","queue_duration_s":2.578399,"scheduling_latency_s":2.578429,"redis_calls":10,"redis_duration_s":0.011196000000000001,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002373,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006397,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002426,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001444876,"duration_s":0.086151,"completed_at":"2026-09-01T08:48:05.842Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004701,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.931951] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.856Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f5b14b01ed11b2b5d5eacef1","created_at":"2026-09-01T08:48:03.191Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4233d91eb5c8da12cee91a107ac091f6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.198Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f5b14b01ed11b2b5d5eacef1: start","job_status":"start","queue_duration_s":2.657509,"scheduling_latency_s":2.657578} gitlab # [ 161.934495] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.877Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 161.960733] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.903Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f5b14b01ed11b2b5d5eacef1","created_at":"2026-09-01T08:48:03.191Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4233d91eb5c8da12cee91a107ac091f6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.198Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f5b14b01ed11b2b5d5eacef1: done: 0.046916 sec","job_status":"done","queue_duration_s":2.657509,"scheduling_latency_s":2.657578,"redis_calls":10,"redis_duration_s":0.002088,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001269,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000404,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000335238,"duration_s":0.046916,"completed_at":"2026-09-01T08:48:05.903Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003944,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 161.971973] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.910Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"5a807d1ffbd5c2fdea606c65","created_at":"2026-09-01T08:48:03.205Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd8c0200254d922036b868d965dc0b1e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.213Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-5a807d1ffbd5c2fdea606c65: start","job_status":"start","queue_duration_s":2.69761,"scheduling_latency_s":2.697642} gitlab # [ 162.044897] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.987Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"5a807d1ffbd5c2fdea606c65","created_at":"2026-09-01T08:48:03.205Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd8c0200254d922036b868d965dc0b1e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.213Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-5a807d1ffbd5c2fdea606c65: done: 0.076992 sec","job_status":"done","queue_duration_s":2.69761,"scheduling_latency_s":2.697642,"redis_calls":19,"redis_duration_s":0.022924,"redis_read_bytes":22,"redis_write_bytes":4777,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00428,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000773,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3007,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.015551,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00232,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000828038,"duration_s":0.076992,"completed_at":"2026-09-01T08:48:05.987Z","load_balancing_strategy":"primary","db_duration_s":0.004953,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.097496] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:05.993Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"7a4ccdeaac442df72a1c512e","created_at":"2026-09-01T08:48:03.221Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b29ba209678949610341348db1466c58","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.228Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-7a4ccdeaac442df72a1c512e: start","job_status":"start","queue_duration_s":2.764474,"scheduling_latency_s":2.764498} gitlab # [ 162.098297] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:06.031Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"7a4ccdeaac442df72a1c512e","created_at":"2026-09-01T08:48:03.221Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b29ba209678949610341348db1466c58","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.228Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-7a4ccdeaac442df72a1c512e: done: 0.037909 sec","job_status":"done","queue_duration_s":2.764474,"scheduling_latency_s":2.764498,"redis_calls":12,"redis_duration_s":0.016453,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003063,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.008999,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004391,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001943264,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.037909,"completed_at":"2026-09-01T08:48:06.031Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001708,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.115381] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:06.035Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"554c78a99910925ebc2c2ec0","created_at":"2026-09-01T08:48:03.243Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4e57a9c1aed2f058f16a8e8df03d640","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.277Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-554c78a99910925ebc2c2ec0: start","job_status":"start","queue_duration_s":2.75806,"scheduling_latency_s":2.758085} gitlab # [ 162.214259] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:06.157Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"554c78a99910925ebc2c2ec0","created_at":"2026-09-01T08:48:03.243Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b4e57a9c1aed2f058f16a8e8df03d640","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.277Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-554c78a99910925ebc2c2ec0: done: 0.121353 sec","job_status":"done","queue_duration_s":2.75806,"scheduling_latency_s":2.758085,"redis_calls":12,"redis_duration_s":0.094178,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001518,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005557,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.087103,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011735,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000866032,"exclusive_lock_hold_duration_s":0.0038222739999999996,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.121353,"completed_at":"2026-09-01T08:48:06.157Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002334,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.277982] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:06.221Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"7476182dd1ec60e4904e71a6","created_at":"2026-09-01T08:48:03.289Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.309Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-7476182dd1ec60e4904e71a6: start","job_status":"start","queue_duration_s":2.911534,"scheduling_latency_s":2.911566} gitlab # [ 162.502148] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:06.445Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"7476182dd1ec60e4904e71a6","created_at":"2026-09-01T08:48:03.289Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.309Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-7476182dd1ec60e4904e71a6: done: 0.224071 sec","job_status":"done","queue_duration_s":2.911534,"scheduling_latency_s":2.911566,"redis_calls":71,"redis_duration_s":0.073274,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.010142,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.002151,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.058865,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002116,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.085232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001630095,"duration_s":0.224071,"completed_at":"2026-09-01T08:48:06.445Z","load_balancing_strategy":"primary","db_duration_s":0.005731,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.556619] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:06.499Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"dd6f61c34af41c69fbca6b72","created_at":"2026-09-01T08:48:03.334Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a029f5563e9b9e0ac10460613e5ea73a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T08:48:03.353Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-dd6f61c34af41c69fbca6b72: start","job_status":"start","queue_duration_s":3.146511,"scheduling_latency_s":3.14654} gitlab # [ 162.600423] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:06.543Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 162.617434] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:06.560Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"dd6f61c34af41c69fbca6b72","created_at":"2026-09-01T08:48:03.334Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a029f5563e9b9e0ac10460613e5ea73a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-01T08:48:03.353Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-dd6f61c34af41c69fbca6b72: done: 0.060572 sec","job_status":"done","queue_duration_s":3.146511,"scheduling_latency_s":3.14654,"redis_calls":10,"redis_duration_s":0.009868,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002114,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005353,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002401,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000608178,"duration_s":0.060572,"completed_at":"2026-09-01T08:48:06.560Z","load_balancing_strategy":"primary","db_duration_s":0.001726,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.648184] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:06.567Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"bc636749d50f6e9606294c88","created_at":"2026-09-01T08:48:03.368Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0385a950293546a557f646c9ba338535","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.386Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-bc636749d50f6e9606294c88: start","job_status":"start","queue_duration_s":3.181221,"scheduling_latency_s":3.18125} gitlab # [ 162.649365] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:06.588Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 162.706846] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:06.649Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"bc636749d50f6e9606294c88","created_at":"2026-09-01T08:48:03.368Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0385a950293546a557f646c9ba338535","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.386Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-bc636749d50f6e9606294c88: done: 0.082446 sec","job_status":"done","queue_duration_s":3.181221,"scheduling_latency_s":3.18125,"redis_calls":10,"redis_duration_s":0.058650999999999995,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001465,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.05515,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002036,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011549,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001713067,"duration_s":0.082446,"completed_at":"2026-09-01T08:48:06.649Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002394,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 162.735061] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:06.665Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"ee9828b2420583a50b3ef40a","created_at":"2026-09-01T08:48:03.415Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.435Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-ee9828b2420583a50b3ef40a: start","job_status":"start","queue_duration_s":3.230116,"scheduling_latency_s":3.230156} gitlab # [ 162.987461] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:06.930Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"ee9828b2420583a50b3ef40a","created_at":"2026-09-01T08:48:03.415Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-ca8a7fd6f8854afc","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:03.435Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-ee9828b2420583a50b3ef40a: done: 0.264453 sec","job_status":"done","queue_duration_s":3.230116,"scheduling_latency_s":3.230156,"redis_calls":65,"redis_duration_s":0.068676,"redis_read_bytes":115,"redis_write_bytes":41282,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004934,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_calls":2,"redis_queues_duration_s":0.001435,"redis_queues_read_bytes":4,"redis_queues_write_bytes":36985,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.058264,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3374,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004043,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.089838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001334527,"duration_s":0.264453,"completed_at":"2026-09-01T08:48:06.930Z","load_balancing_strategy":"primary","db_duration_s":0.001999,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.041856] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:06.933Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b97907c22acce587717ba446","created_at":"2026-09-01T08:48:05.944Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-28ed8545b9bc427d","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd8c0200254d922036b868d965dc0b1e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:05.972Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b97907c22acce587717ba446: start","job_status":"start","queue_duration_s":0.960524,"scheduling_latency_s":0.960556} gitlab # [ 163.065066] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.008Z","message":"No existing merge request to be cleaned up."} gitlab # [ 163.087355] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.030Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b97907c22acce587717ba446","created_at":"2026-09-01T08:48:05.944Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-28ed8545b9bc427d","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd8c0200254d922036b868d965dc0b1e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:05.972Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b97907c22acce587717ba446: done: 0.097044 sec","job_status":"done","queue_duration_s":0.960524,"scheduling_latency_s":0.960556,"redis_calls":13,"redis_duration_s":0.008458,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002057,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003717,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002684,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.051,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014453,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00129821,"duration_s":0.097044,"completed_at":"2026-09-01T08:48:07.030Z","load_balancing_strategy":"primary","db_duration_s":0.086785,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.148829] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.091Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6222fe24f36f9de6b92a284f","created_at":"2026-09-01T08:48:05.944Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-28ed8545b9bc427d","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd8c0200254d922036b868d965dc0b1e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:05.972Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6222fe24f36f9de6b92a284f: start","job_status":"start","queue_duration_s":1.118878,"scheduling_latency_s":1.118912} gitlab # [ 163.180494] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.123Z","message":"No existing merge request to be cleaned up."} gitlab # [ 163.191212] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.133Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6222fe24f36f9de6b92a284f","created_at":"2026-09-01T08:48:05.944Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-28ed8545b9bc427d","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd8c0200254d922036b868d965dc0b1e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:05.972Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6222fe24f36f9de6b92a284f: done: 0.042476 sec","job_status":"done","queue_duration_s":1.118878,"scheduling_latency_s":1.118912,"redis_calls":10,"redis_duration_s":0.004824999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000893,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003932,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042476,"completed_at":"2026-09-01T08:48:07.133Z","load_balancing_strategy":"primary","db_duration_s":0.001896,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.224965] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.136Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fdc11876bb35ccae87db21e5","created_at":"2026-09-01T08:48:05.944Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-28ed8545b9bc427d","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd8c0200254d922036b868d965dc0b1e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:05.972Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fdc11876bb35ccae87db21e5: start","job_status":"start","queue_duration_s":1.163538,"scheduling_latency_s":1.163566} gitlab # [ 163.226142] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.148Z","message":"No existing merge request to be cleaned up."} gitlab # [ 163.226990] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.154Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fdc11876bb35ccae87db21e5","created_at":"2026-09-01T08:48:05.944Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-28ed8545b9bc427d","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd8c0200254d922036b868d965dc0b1e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:05.972Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fdc11876bb35ccae87db21e5: done: 0.018602 sec","job_status":"done","queue_duration_s":1.163538,"scheduling_latency_s":1.163566,"redis_calls":10,"redis_duration_s":0.0025080000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000885,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001623,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018602,"completed_at":"2026-09-01T08:48:07.154Z","load_balancing_strategy":"primary","db_duration_s":0.002432,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.283763] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.156Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c91c0ef6c8cd3f1af6d6902c","created_at":"2026-09-01T08:48:05.944Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-28ed8545b9bc427d","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd8c0200254d922036b868d965dc0b1e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:05.972Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c91c0ef6c8cd3f1af6d6902c: start","job_status":"start","queue_duration_s":1.183525,"scheduling_latency_s":1.183552} gitlab # [ 163.323669] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.266Z","message":"No existing merge request to be cleaned up."} gitlab # [ 163.333478] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.276Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c91c0ef6c8cd3f1af6d6902c","created_at":"2026-09-01T08:48:05.944Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-28ed8545b9bc427d","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd8c0200254d922036b868d965dc0b1e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:05.972Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c91c0ef6c8cd3f1af6d6902c: done: 0.120156 sec","job_status":"done","queue_duration_s":1.183525,"scheduling_latency_s":1.183552,"redis_calls":10,"redis_duration_s":0.004332,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003405,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.101,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.101,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.120156,"completed_at":"2026-09-01T08:48:07.276Z","load_balancing_strategy":"primary","db_duration_s":0.201101,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.378422] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3fc324ee618874a76d5946d","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3fc324ee618874a76d5946d: start","job_status":"start","queue_duration_s":0.891952,"scheduling_latency_s":0.891984} gitlab # [ 163.415610] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3fc324ee618874a76d5946d","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3fc324ee618874a76d5946d: done: 0.037224 sec","job_status":"done","queue_duration_s":0.891952,"scheduling_latency_s":0.891984,"redis_calls":16,"redis_duration_s":0.003426,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00044,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001127,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001859,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.003451,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000382731,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.037224,"completed_at":"2026-09-01T08:48:07.358Z","load_balancing_strategy":"primary","db_duration_s":0.000536,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.464834] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d0ce96bb9e606531f5ca4bb","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d0ce96bb9e606531f5ca4bb: start","job_status":"start","queue_duration_s":0.978296,"scheduling_latency_s":0.978332} gitlab # [ 163.486645] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d0ce96bb9e606531f5ca4bb","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d0ce96bb9e606531f5ca4bb: done: 0.021883 sec","job_status":"done","queue_duration_s":0.978296,"scheduling_latency_s":0.978332,"redis_calls":13,"redis_duration_s":0.003554,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002554,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021883,"completed_at":"2026-09-01T08:48:07.429Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.541307] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03fc00a655b9e2f9e9051640","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03fc00a655b9e2f9e9051640: start","job_status":"start","queue_duration_s":1.001659,"scheduling_latency_s":1.001687} gitlab # [ 163.542229] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03fc00a655b9e2f9e9051640","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03fc00a655b9e2f9e9051640: done: 0.019078 sec","job_status":"done","queue_duration_s":1.001659,"scheduling_latency_s":1.001687,"redis_calls":13,"redis_duration_s":0.003102,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019078,"completed_at":"2026-09-01T08:48:07.450Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.549751] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43228c6f0f9372b0873145cd","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43228c6f0f9372b0873145cd: start","job_status":"start","queue_duration_s":1.022511,"scheduling_latency_s":1.022539} gitlab # [ 163.566083] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.472Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43228c6f0f9372b0873145cd","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43228c6f0f9372b0873145cd: done: 0.020228 sec","job_status":"done","queue_duration_s":1.022511,"scheduling_latency_s":1.022539,"redis_calls":13,"redis_duration_s":0.003585,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001111,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002474,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020228,"completed_at":"2026-09-01T08:48:07.472Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.624880] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d9c0f22834a14317787aa58","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d9c0f22834a14317787aa58: start","job_status":"start","queue_duration_s":1.04443,"scheduling_latency_s":1.044457} gitlab # [ 163.693432] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d9c0f22834a14317787aa58","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d9c0f22834a14317787aa58: done: 0.162334 sec","job_status":"done","queue_duration_s":1.04443,"scheduling_latency_s":1.044457,"redis_calls":13,"redis_duration_s":0.146032,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001097,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.144935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.162334,"completed_at":"2026-09-01T08:48:07.636Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.742919] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.685Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"502a9fb56705bd0809f865f9","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-502a9fb56705bd0809f865f9: start","job_status":"start","queue_duration_s":1.256548,"scheduling_latency_s":1.256582} gitlab # [ 163.761421] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"502a9fb56705bd0809f865f9","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-502a9fb56705bd0809f865f9: done: 0.018474 sec","job_status":"done","queue_duration_s":1.256548,"scheduling_latency_s":1.256582,"redis_calls":13,"redis_duration_s":0.003037,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018474,"completed_at":"2026-09-01T08:48:07.704Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.815707] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"137829395f5e6cd4f5a482aa","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-137829395f5e6cd4f5a482aa: start","job_status":"start","queue_duration_s":1.329273,"scheduling_latency_s":1.329307} gitlab # [ 163.835329] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"137829395f5e6cd4f5a482aa","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-137829395f5e6cd4f5a482aa: done: 0.017115 sec","job_status":"done","queue_duration_s":1.329273,"scheduling_latency_s":1.329307,"redis_calls":13,"redis_duration_s":0.002807,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017115,"completed_at":"2026-09-01T08:48:07.775Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.836948] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d46a3668c43e88e0a3f04bb","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d46a3668c43e88e0a3f04bb: start","job_status":"start","queue_duration_s":1.347658,"scheduling_latency_s":1.347677} gitlab # [ 163.929748] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d46a3668c43e88e0a3f04bb","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d46a3668c43e88e0a3f04bb: done: 0.095554 sec","job_status":"done","queue_duration_s":1.347658,"scheduling_latency_s":1.347677,"redis_calls":13,"redis_duration_s":0.002863,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.095554,"completed_at":"2026-09-01T08:48:07.872Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 163.931538] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a6acfc2f89cd1213eeb1a2a","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a6acfc2f89cd1213eeb1a2a: start","job_status":"start","queue_duration_s":1.445209,"scheduling_latency_s":1.445252} gitlab # [ 163.952189] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2a6acfc2f89cd1213eeb1a2a","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2a6acfc2f89cd1213eeb1a2a: done: 0.020213 sec","job_status":"done","queue_duration_s":1.445209,"scheduling_latency_s":1.445252,"redis_calls":13,"redis_duration_s":0.0032119999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000978,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002234,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020213,"completed_at":"2026-09-01T08:48:07.894Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.029941] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b53e7895a492d6dc1d4c9137","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b53e7895a492d6dc1d4c9137: start","job_status":"start","queue_duration_s":1.46741,"scheduling_latency_s":1.467441} gitlab # [ 164.031142] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b53e7895a492d6dc1d4c9137","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b53e7895a492d6dc1d4c9137: done: 0.019119 sec","job_status":"done","queue_duration_s":1.46741,"scheduling_latency_s":1.467441,"redis_calls":13,"redis_duration_s":0.002974,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019119,"completed_at":"2026-09-01T08:48:07.915Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.109262] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed52582dfe5c0f739a33877d","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed52582dfe5c0f739a33877d: start","job_status":"start","queue_duration_s":1.488243,"scheduling_latency_s":1.48827} gitlab # [ 164.110440] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.935Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed52582dfe5c0f739a33877d","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed52582dfe5c0f739a33877d: done: 0.017477 sec","job_status":"done","queue_duration_s":1.488243,"scheduling_latency_s":1.48827,"redis_calls":13,"redis_duration_s":0.0030499999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001024,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002026,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017477,"completed_at":"2026-09-01T08:48:07.935Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.200556] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1457026396e019bdfbe4bfd0","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1457026396e019bdfbe4bfd0: start","job_status":"start","queue_duration_s":1.507349,"scheduling_latency_s":1.507371} gitlab # [ 164.202044] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1457026396e019bdfbe4bfd0","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1457026396e019bdfbe4bfd0: done: 0.015278 sec","job_status":"done","queue_duration_s":1.507349,"scheduling_latency_s":1.507371,"redis_calls":13,"redis_duration_s":0.003078,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002128,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015278,"completed_at":"2026-09-01T08:48:07.951Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.281848] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.953Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b12afa7e5bd293d1f7984e5","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b12afa7e5bd293d1f7984e5: start","job_status":"start","queue_duration_s":1.524265,"scheduling_latency_s":1.524299} gitlab # [ 164.300255] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.971Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b12afa7e5bd293d1f7984e5","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b12afa7e5bd293d1f7984e5: done: 0.017408 sec","job_status":"done","queue_duration_s":1.524265,"scheduling_latency_s":1.524299,"redis_calls":13,"redis_duration_s":0.003119,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002204,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017408,"completed_at":"2026-09-01T08:48:07.970Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.329705] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c649ef39f15cafdddc22acfc","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c649ef39f15cafdddc22acfc: start","job_status":"start","queue_duration_s":1.543207,"scheduling_latency_s":1.543229} gitlab # [ 164.330892] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.988Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c649ef39f15cafdddc22acfc","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c649ef39f15cafdddc22acfc: done: 0.016027 sec","job_status":"done","queue_duration_s":1.543207,"scheduling_latency_s":1.543229,"redis_calls":13,"redis_duration_s":0.002417,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000763,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001654,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016027,"completed_at":"2026-09-01T08:48:07.988Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.387445] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:07.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24a1139d3c0efa6c3ca055b8","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24a1139d3c0efa6c3ca055b8: start","job_status":"start","queue_duration_s":1.561087,"scheduling_latency_s":1.561113} gitlab # [ 164.388439] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24a1139d3c0efa6c3ca055b8","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24a1139d3c0efa6c3ca055b8: done: 0.020086 sec","job_status":"done","queue_duration_s":1.561087,"scheduling_latency_s":1.561113,"redis_calls":13,"redis_duration_s":0.002779,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000832,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001947,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020086,"completed_at":"2026-09-01T08:48:08.010Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.446723] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da66d7bcba130e9d7299e72c","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da66d7bcba130e9d7299e72c: start","job_status":"start","queue_duration_s":1.582815,"scheduling_latency_s":1.582844} gitlab # [ 164.448143] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da66d7bcba130e9d7299e72c","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da66d7bcba130e9d7299e72c: done: 0.021256 sec","job_status":"done","queue_duration_s":1.582815,"scheduling_latency_s":1.582844,"redis_calls":13,"redis_duration_s":0.002902,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021256,"completed_at":"2026-09-01T08:48:08.033Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.489753] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0026ea2d9450dee6b089a30e","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0026ea2d9450dee6b089a30e: start","job_status":"start","queue_duration_s":1.605808,"scheduling_latency_s":1.605835} gitlab # [ 164.566146] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0026ea2d9450dee6b089a30e","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0026ea2d9450dee6b089a30e: done: 0.019934 sec","job_status":"done","queue_duration_s":1.605808,"scheduling_latency_s":1.605835,"redis_calls":13,"redis_duration_s":0.0036509999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001017,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002634,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019934,"completed_at":"2026-09-01T08:48:08.055Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.568139] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c31b9bae358c3b0fb5f97cbd","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c31b9bae358c3b0fb5f97cbd: start","job_status":"start","queue_duration_s":1.627199,"scheduling_latency_s":1.627221} gitlab # [ 164.634264] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c31b9bae358c3b0fb5f97cbd","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c31b9bae358c3b0fb5f97cbd: done: 0.01571 sec","job_status":"done","queue_duration_s":1.627199,"scheduling_latency_s":1.627221,"redis_calls":13,"redis_duration_s":0.002427,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000797,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01571,"completed_at":"2026-09-01T08:48:08.072Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.640037] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95508d0f8c02cc79e455f176","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95508d0f8c02cc79e455f176: start","job_status":"start","queue_duration_s":1.644386,"scheduling_latency_s":1.644409} gitlab # [ 164.700167] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95508d0f8c02cc79e455f176","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95508d0f8c02cc79e455f176: done: 0.015654 sec","job_status":"done","queue_duration_s":1.644386,"scheduling_latency_s":1.644409,"redis_calls":13,"redis_duration_s":0.002663,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000852,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001811,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015654,"completed_at":"2026-09-01T08:48:08.089Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.703147] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e4a4692f9c65682257ab7dd","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e4a4692f9c65682257ab7dd: start","job_status":"start","queue_duration_s":1.661499,"scheduling_latency_s":1.66152} gitlab # [ 164.720927] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.108Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e4a4692f9c65682257ab7dd","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e4a4692f9c65682257ab7dd: done: 0.017242 sec","job_status":"done","queue_duration_s":1.661499,"scheduling_latency_s":1.66152,"redis_calls":13,"redis_duration_s":0.00286,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017242,"completed_at":"2026-09-01T08:48:08.108Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.777400] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02da256d9f521db40b5586af","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02da256d9f521db40b5586af: start","job_status":"start","queue_duration_s":1.680411,"scheduling_latency_s":1.680437} gitlab # [ 164.781727] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02da256d9f521db40b5586af","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02da256d9f521db40b5586af: done: 0.017876 sec","job_status":"done","queue_duration_s":1.680411,"scheduling_latency_s":1.680437,"redis_calls":13,"redis_duration_s":0.0030080000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001011,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017876,"completed_at":"2026-09-01T08:48:08.127Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.852959] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.129Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d5331c3fc0fa69335cb7bf2","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d5331c3fc0fa69335cb7bf2: start","job_status":"start","queue_duration_s":1.699764,"scheduling_latency_s":1.699786} gitlab # [ 164.854073] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.145Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d5331c3fc0fa69335cb7bf2","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d5331c3fc0fa69335cb7bf2: done: 0.015907 sec","job_status":"done","queue_duration_s":1.699764,"scheduling_latency_s":1.699786,"redis_calls":13,"redis_duration_s":0.002794,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001863,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015907,"completed_at":"2026-09-01T08:48:08.145Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.890512] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8033663d7b3627c9d919cf98","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8033663d7b3627c9d919cf98: start","job_status":"start","queue_duration_s":1.718176,"scheduling_latency_s":1.71821} gitlab # [ 164.948851] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8033663d7b3627c9d919cf98","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8033663d7b3627c9d919cf98: done: 0.017641 sec","job_status":"done","queue_duration_s":1.718176,"scheduling_latency_s":1.71821,"redis_calls":13,"redis_duration_s":0.002851,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000866,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017641,"completed_at":"2026-09-01T08:48:08.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.950640] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"528a33485a268e6899a2fa2b","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-528a33485a268e6899a2fa2b: start","job_status":"start","queue_duration_s":1.737209,"scheduling_latency_s":1.73723} gitlab # [ 164.965371] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"528a33485a268e6899a2fa2b","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-528a33485a268e6899a2fa2b: done: 0.014478 sec","job_status":"done","queue_duration_s":1.737209,"scheduling_latency_s":1.73723,"redis_calls":13,"redis_duration_s":0.00244,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000745,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001695,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014478,"completed_at":"2026-09-01T08:48:08.181Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.026935] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ce96a7f1741c5519f83de5d","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ce96a7f1741c5519f83de5d: start","job_status":"start","queue_duration_s":1.753095,"scheduling_latency_s":1.753114} gitlab # [ 165.027930] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.197Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ce96a7f1741c5519f83de5d","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ce96a7f1741c5519f83de5d: done: 0.014679 sec","job_status":"done","queue_duration_s":1.753095,"scheduling_latency_s":1.753114,"redis_calls":13,"redis_duration_s":0.002499,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000794,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001705,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014679,"completed_at":"2026-09-01T08:48:08.197Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.119352] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f45f0b1f4f86275d9d173ad4","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f45f0b1f4f86275d9d173ad4: start","job_status":"start","queue_duration_s":1.769315,"scheduling_latency_s":1.769336} gitlab # [ 165.120426] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.213Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f45f0b1f4f86275d9d173ad4","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f45f0b1f4f86275d9d173ad4: done: 0.014423 sec","job_status":"done","queue_duration_s":1.769315,"scheduling_latency_s":1.769336,"redis_calls":13,"redis_duration_s":0.002366,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000754,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001612,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014423,"completed_at":"2026-09-01T08:48:08.213Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.177188] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.214Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba43c8a26cad3d00f40b1eaa","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba43c8a26cad3d00f40b1eaa: start","job_status":"start","queue_duration_s":1.785114,"scheduling_latency_s":1.785135} gitlab # [ 165.178201] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba43c8a26cad3d00f40b1eaa","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba43c8a26cad3d00f40b1eaa: done: 0.019813 sec","job_status":"done","queue_duration_s":1.785114,"scheduling_latency_s":1.785135,"redis_calls":13,"redis_duration_s":0.00362,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001262,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002358,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019813,"completed_at":"2026-09-01T08:48:08.234Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.230302] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0ea546aed9c9189351e3c38","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0ea546aed9c9189351e3c38: start","job_status":"start","queue_duration_s":1.806475,"scheduling_latency_s":1.806499} gitlab # [ 165.231234] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0ea546aed9c9189351e3c38","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0ea546aed9c9189351e3c38: done: 0.017796 sec","job_status":"done","queue_duration_s":1.806475,"scheduling_latency_s":1.806499,"redis_calls":13,"redis_duration_s":0.0028520000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011665,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017796,"completed_at":"2026-09-01T08:48:08.253Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.254297] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"873e1a17a23ed632cddfe18c","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-873e1a17a23ed632cddfe18c: start","job_status":"start","queue_duration_s":1.826067,"scheduling_latency_s":1.82609} gitlab # [ 165.269042] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"873e1a17a23ed632cddfe18c","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-873e1a17a23ed632cddfe18c: done: 0.019009 sec","job_status":"done","queue_duration_s":1.826067,"scheduling_latency_s":1.82609,"redis_calls":13,"redis_duration_s":0.00335,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001003,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002347,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019009,"completed_at":"2026-09-01T08:48:08.274Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.324924] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.276Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc90679f6c26a82355e8671e","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc90679f6c26a82355e8671e: start","job_status":"start","queue_duration_s":1.846627,"scheduling_latency_s":1.846653} gitlab # [ 165.384188] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc90679f6c26a82355e8671e","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc90679f6c26a82355e8671e: done: 0.01674 sec","job_status":"done","queue_duration_s":1.846627,"scheduling_latency_s":1.846653,"redis_calls":13,"redis_duration_s":0.002772,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001882,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01674,"completed_at":"2026-09-01T08:48:08.292Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.385657] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.294Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c039da7e50af0e4b22972e9c","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c039da7e50af0e4b22972e9c: start","job_status":"start","queue_duration_s":1.864808,"scheduling_latency_s":1.864831} gitlab # [ 165.396093] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.310Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c039da7e50af0e4b22972e9c","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c039da7e50af0e4b22972e9c: done: 0.015842 sec","job_status":"done","queue_duration_s":1.864808,"scheduling_latency_s":1.864831,"redis_calls":13,"redis_duration_s":0.002646,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001842,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010315,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015842,"completed_at":"2026-09-01T08:48:08.309Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.447642] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.311Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1bfaa99b56f40c263fca68e","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1bfaa99b56f40c263fca68e: start","job_status":"start","queue_duration_s":1.881981,"scheduling_latency_s":1.882003} gitlab # [ 165.499303] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d1bfaa99b56f40c263fca68e","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d1bfaa99b56f40c263fca68e: done: 0.015965 sec","job_status":"done","queue_duration_s":1.881981,"scheduling_latency_s":1.882003,"redis_calls":13,"redis_duration_s":0.002738,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000965,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001773,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015965,"completed_at":"2026-09-01T08:48:08.327Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.500951] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.328Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ed32f3e23e9ab63886adb14","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ed32f3e23e9ab63886adb14: start","job_status":"start","queue_duration_s":1.899554,"scheduling_latency_s":1.899578} gitlab # [ 165.516592] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ed32f3e23e9ab63886adb14","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ed32f3e23e9ab63886adb14: done: 0.015839 sec","job_status":"done","queue_duration_s":1.899554,"scheduling_latency_s":1.899578,"redis_calls":13,"redis_duration_s":0.002452,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000767,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001685,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015839,"completed_at":"2026-09-01T08:48:08.344Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.586206] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f99479e04181b461de855bf","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f99479e04181b461de855bf: start","job_status":"start","queue_duration_s":1.917019,"scheduling_latency_s":1.917068} gitlab # [ 165.587250] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f99479e04181b461de855bf","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f99479e04181b461de855bf: done: 0.0166 sec","job_status":"done","queue_duration_s":1.917019,"scheduling_latency_s":1.917068,"redis_calls":13,"redis_duration_s":0.0028409999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001041,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0166,"completed_at":"2026-09-01T08:48:08.362Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.639406] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b94bcc11d5539ecdbf89fce7","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b94bcc11d5539ecdbf89fce7: start","job_status":"start","queue_duration_s":1.935215,"scheduling_latency_s":1.935244} gitlab # [ 165.640438] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b94bcc11d5539ecdbf89fce7","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b94bcc11d5539ecdbf89fce7: done: 0.015646 sec","job_status":"done","queue_duration_s":1.935215,"scheduling_latency_s":1.935244,"redis_calls":13,"redis_duration_s":0.002468,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000785,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001683,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015646,"completed_at":"2026-09-01T08:48:08.380Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.718209] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2461de91e124e949cb1f8d0c","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2461de91e124e949cb1f8d0c: start","job_status":"start","queue_duration_s":1.952337,"scheduling_latency_s":1.952359} gitlab # [ 165.719342] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2461de91e124e949cb1f8d0c","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2461de91e124e949cb1f8d0c: done: 0.015551 sec","job_status":"done","queue_duration_s":1.952337,"scheduling_latency_s":1.952359,"redis_calls":13,"redis_duration_s":0.002466,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001678,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010459,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015551,"completed_at":"2026-09-01T08:48:08.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.786075] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"866f10b550accb62bf9cba82","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-866f10b550accb62bf9cba82: start","job_status":"start","queue_duration_s":1.969558,"scheduling_latency_s":1.969582} gitlab # [ 165.794611] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"866f10b550accb62bf9cba82","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-866f10b550accb62bf9cba82: done: 0.016423 sec","job_status":"done","queue_duration_s":1.969558,"scheduling_latency_s":1.969582,"redis_calls":13,"redis_duration_s":0.002612,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000818,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001794,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016423,"completed_at":"2026-09-01T08:48:08.415Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.857278] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"694e04790cc35b3d45157325","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-694e04790cc35b3d45157325: start","job_status":"start","queue_duration_s":1.987546,"scheduling_latency_s":1.98757} gitlab # [ 165.858531] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"694e04790cc35b3d45157325","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-694e04790cc35b3d45157325: done: 0.016129 sec","job_status":"done","queue_duration_s":1.987546,"scheduling_latency_s":1.98757,"redis_calls":13,"redis_duration_s":0.002539,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000769,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00177,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016129,"completed_at":"2026-09-01T08:48:08.433Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.931081] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d98179ab6c40836a817d9e39","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d98179ab6c40836a817d9e39: start","job_status":"start","queue_duration_s":2.005218,"scheduling_latency_s":2.005252} gitlab # [ 165.932395] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d98179ab6c40836a817d9e39","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d98179ab6c40836a817d9e39: done: 0.016724 sec","job_status":"done","queue_duration_s":2.005218,"scheduling_latency_s":2.005252,"redis_calls":13,"redis_duration_s":0.002686,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001817,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016724,"completed_at":"2026-09-01T08:48:08.451Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.988283] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94a1ab50d8040c26cacf70c8","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94a1ab50d8040c26cacf70c8: start","job_status":"start","queue_duration_s":2.023561,"scheduling_latency_s":2.023586} gitlab # [ 165.989322] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"94a1ab50d8040c26cacf70c8","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-94a1ab50d8040c26cacf70c8: done: 0.017164 sec","job_status":"done","queue_duration_s":2.023561,"scheduling_latency_s":2.023586,"redis_calls":13,"redis_duration_s":0.0029879999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002129,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017164,"completed_at":"2026-09-01T08:48:08.470Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.023826] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.471Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30e7e1175db478ae2d2dd34b","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30e7e1175db478ae2d2dd34b: start","job_status":"start","queue_duration_s":2.042247,"scheduling_latency_s":2.042272} gitlab # [ 166.074422] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30e7e1175db478ae2d2dd34b","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30e7e1175db478ae2d2dd34b: done: 0.017077 sec","job_status":"done","queue_duration_s":2.042247,"scheduling_latency_s":2.042272,"redis_calls":13,"redis_duration_s":0.0026709999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001803,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017077,"completed_at":"2026-09-01T08:48:08.488Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.082954] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4247833770fac1b91026f555","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4247833770fac1b91026f555: start","job_status":"start","queue_duration_s":2.0612,"scheduling_latency_s":2.061227} gitlab # [ 166.097918] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4247833770fac1b91026f555","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4247833770fac1b91026f555: done: 0.017924 sec","job_status":"done","queue_duration_s":2.0612,"scheduling_latency_s":2.061227,"redis_calls":13,"redis_duration_s":0.002762,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000878,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017924,"completed_at":"2026-09-01T08:48:08.508Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.176746] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"155ca27dc0e2b76a9185b7df","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-155ca27dc0e2b76a9185b7df: start","job_status":"start","queue_duration_s":2.080768,"scheduling_latency_s":2.080794} gitlab # [ 166.178062] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.527Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"155ca27dc0e2b76a9185b7df","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-155ca27dc0e2b76a9185b7df: done: 0.0177 sec","job_status":"done","queue_duration_s":2.080768,"scheduling_latency_s":2.080794,"redis_calls":13,"redis_duration_s":0.002831,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001885,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011169,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0177,"completed_at":"2026-09-01T08:48:08.527Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.257547] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d7a76cb837bc393bf58d82b","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d7a76cb837bc393bf58d82b: start","job_status":"start","queue_duration_s":2.100339,"scheduling_latency_s":2.100364} gitlab # [ 166.258967] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.547Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d7a76cb837bc393bf58d82b","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d7a76cb837bc393bf58d82b: done: 0.017718 sec","job_status":"done","queue_duration_s":2.100339,"scheduling_latency_s":2.100364,"redis_calls":13,"redis_duration_s":0.003057,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000851,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017718,"completed_at":"2026-09-01T08:48:08.547Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.276911] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b67b4568fc22ecf6239559b","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b67b4568fc22ecf6239559b: start","job_status":"start","queue_duration_s":2.119669,"scheduling_latency_s":2.119691} gitlab # [ 166.296149] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.566Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b67b4568fc22ecf6239559b","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b67b4568fc22ecf6239559b: done: 0.017614 sec","job_status":"done","queue_duration_s":2.119669,"scheduling_latency_s":2.119691,"redis_calls":13,"redis_duration_s":0.0031859999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000812,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002374,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017614,"completed_at":"2026-09-01T08:48:08.566Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.369068] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9f7cf979bf8996f6b38b2b7","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9f7cf979bf8996f6b38b2b7: start","job_status":"start","queue_duration_s":2.138786,"scheduling_latency_s":2.138809} gitlab # [ 166.370170] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9f7cf979bf8996f6b38b2b7","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9f7cf979bf8996f6b38b2b7: done: 0.014318 sec","job_status":"done","queue_duration_s":2.138786,"scheduling_latency_s":2.138809,"redis_calls":13,"redis_duration_s":0.002573,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001832,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014318,"completed_at":"2026-09-01T08:48:08.582Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.428742] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de0626e5e9ae7b459afecde2","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de0626e5e9ae7b459afecde2: start","job_status":"start","queue_duration_s":2.154499,"scheduling_latency_s":2.154517} gitlab # [ 166.433359] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de0626e5e9ae7b459afecde2","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de0626e5e9ae7b459afecde2: done: 0.017663 sec","job_status":"done","queue_duration_s":2.154499,"scheduling_latency_s":2.154517,"redis_calls":13,"redis_duration_s":0.002617,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000794,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001823,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017663,"completed_at":"2026-09-01T08:48:08.601Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.485933] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78dc74c2a66e37df3726509d","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78dc74c2a66e37df3726509d: start","job_status":"start","queue_duration_s":2.173491,"scheduling_latency_s":2.173512} gitlab # [ 166.492790] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78dc74c2a66e37df3726509d","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78dc74c2a66e37df3726509d: done: 0.061756 sec","job_status":"done","queue_duration_s":2.173491,"scheduling_latency_s":2.173512,"redis_calls":13,"redis_duration_s":0.003235,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00233,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.061756,"completed_at":"2026-09-01T08:48:08.664Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.561773] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76846ec935a0fe5629cc7e36","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76846ec935a0fe5629cc7e36: start","job_status":"start","queue_duration_s":2.237633,"scheduling_latency_s":2.237662} gitlab # [ 166.563210] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76846ec935a0fe5629cc7e36","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76846ec935a0fe5629cc7e36: done: 0.017934 sec","job_status":"done","queue_duration_s":2.237633,"scheduling_latency_s":2.237662,"redis_calls":13,"redis_duration_s":0.002782,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000885,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017934,"completed_at":"2026-09-01T08:48:08.684Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.632108] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"baaf5a82d88463761bd3f22d","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-baaf5a82d88463761bd3f22d: start","job_status":"start","queue_duration_s":2.257196,"scheduling_latency_s":2.257226} gitlab # [ 166.633146] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"baaf5a82d88463761bd3f22d","created_at":"2026-09-01T08:48:06.254Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-265a27b529484dce","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"b40850113153dca7ae8d7fd281ba09b6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.429Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-baaf5a82d88463761bd3f22d: done: 0.016415 sec","job_status":"done","queue_duration_s":2.257196,"scheduling_latency_s":2.257226,"redis_calls":13,"redis_duration_s":0.003164,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001272,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001892,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016415,"completed_at":"2026-09-01T08:48:08.702Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.696947] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3e9cf08d571a657575fe0fdc","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3e9cf08d571a657575fe0fdc: start","job_status":"start","queue_duration_s":1.789171,"scheduling_latency_s":1.789199} gitlab # [ 166.697925] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3e9cf08d571a657575fe0fdc","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3e9cf08d571a657575fe0fdc: done: 0.031329 sec","job_status":"done","queue_duration_s":1.789171,"scheduling_latency_s":1.789199,"redis_calls":13,"redis_duration_s":0.003092,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000451,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001164,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001477,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000372115,"duration_s":0.031329,"completed_at":"2026-09-01T08:48:08.735Z","load_balancing_strategy":"primary","db_duration_s":0.005393,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.760371] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b85a1360bb616acd3a4b2213","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b85a1360bb616acd3a4b2213: start","job_status":"start","queue_duration_s":1.821992,"scheduling_latency_s":1.822019} gitlab # [ 166.761628] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b85a1360bb616acd3a4b2213","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b85a1360bb616acd3a4b2213: done: 0.018652 sec","job_status":"done","queue_duration_s":1.821992,"scheduling_latency_s":1.822019,"redis_calls":10,"redis_duration_s":0.00228,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001332,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018652,"completed_at":"2026-09-01T08:48:08.755Z","load_balancing_strategy":"primary","db_duration_s":0.001906,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.808142] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9181dba90c73ed68fd1323f7","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9181dba90c73ed68fd1323f7: start","job_status":"start","queue_duration_s":1.842049,"scheduling_latency_s":1.842074} gitlab # [ 166.869138] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9181dba90c73ed68fd1323f7","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9181dba90c73ed68fd1323f7: done: 0.018721 sec","job_status":"done","queue_duration_s":1.842049,"scheduling_latency_s":1.842074,"redis_calls":10,"redis_duration_s":0.002584,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001161,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001423,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018721,"completed_at":"2026-09-01T08:48:08.776Z","load_balancing_strategy":"primary","db_duration_s":0.001886,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.870118] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0ac7df894b3e2e7b53a565bf","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0ac7df894b3e2e7b53a565bf: start","job_status":"start","queue_duration_s":1.862715,"scheduling_latency_s":1.86274} gitlab # [ 166.937262] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0ac7df894b3e2e7b53a565bf","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0ac7df894b3e2e7b53a565bf: done: 0.018403 sec","job_status":"done","queue_duration_s":1.862715,"scheduling_latency_s":1.86274,"redis_calls":10,"redis_duration_s":0.002232,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001322,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018403,"completed_at":"2026-09-01T08:48:08.796Z","load_balancing_strategy":"primary","db_duration_s":0.00223,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.939061] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"61225ec9fc1a8248a01d9a57","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-61225ec9fc1a8248a01d9a57: start","job_status":"start","queue_duration_s":1.882611,"scheduling_latency_s":1.882636} gitlab # [ 166.999794] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"61225ec9fc1a8248a01d9a57","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-61225ec9fc1a8248a01d9a57: done: 0.023196 sec","job_status":"done","queue_duration_s":1.882611,"scheduling_latency_s":1.882636,"redis_calls":10,"redis_duration_s":0.002355,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000841,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001514,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023196,"completed_at":"2026-09-01T08:48:08.821Z","load_balancing_strategy":"primary","db_duration_s":0.008037,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.001377] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1c79508514e5c102f83d6049","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1c79508514e5c102f83d6049: start","job_status":"start","queue_duration_s":1.907704,"scheduling_latency_s":1.90773} gitlab # [ 167.011927] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1c79508514e5c102f83d6049","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1c79508514e5c102f83d6049: done: 0.019889 sec","job_status":"done","queue_duration_s":1.907704,"scheduling_latency_s":1.90773,"redis_calls":10,"redis_duration_s":0.002187,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000845,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001342,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019889,"completed_at":"2026-09-01T08:48:08.842Z","load_balancing_strategy":"primary","db_duration_s":0.003291,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.087966] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"27cd47f4bec69d5064e058c8","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-27cd47f4bec69d5064e058c8: start","job_status":"start","queue_duration_s":1.929075,"scheduling_latency_s":1.929098} gitlab # [ 167.088983] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"27cd47f4bec69d5064e058c8","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-27cd47f4bec69d5064e058c8: done: 0.018537 sec","job_status":"done","queue_duration_s":1.929075,"scheduling_latency_s":1.929098,"redis_calls":10,"redis_duration_s":0.0024089999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001514,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018537,"completed_at":"2026-09-01T08:48:08.862Z","load_balancing_strategy":"primary","db_duration_s":0.001833,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.162990] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a542d7f80b587dfc945c5ae8","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a542d7f80b587dfc945c5ae8: start","job_status":"start","queue_duration_s":1.949219,"scheduling_latency_s":1.949244} gitlab # [ 167.164094] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a542d7f80b587dfc945c5ae8","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a542d7f80b587dfc945c5ae8: done: 0.01879 sec","job_status":"done","queue_duration_s":1.949219,"scheduling_latency_s":1.949244,"redis_calls":10,"redis_duration_s":0.002251,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001369,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01879,"completed_at":"2026-09-01T08:48:08.883Z","load_balancing_strategy":"primary","db_duration_s":0.001795,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.176951] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"213449e5d41a232bad1c1ce6","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-213449e5d41a232bad1c1ce6: start","job_status":"start","queue_duration_s":1.969491,"scheduling_latency_s":1.969517} gitlab # [ 167.191894] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"213449e5d41a232bad1c1ce6","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-213449e5d41a232bad1c1ce6: done: 0.022377 sec","job_status":"done","queue_duration_s":1.969491,"scheduling_latency_s":1.969517,"redis_calls":10,"redis_duration_s":0.002433,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001524,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022377,"completed_at":"2026-09-01T08:48:08.907Z","load_balancing_strategy":"primary","db_duration_s":0.006939,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.245554] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8bcaf3cc264e7f3f7008e152","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8bcaf3cc264e7f3f7008e152: start","job_status":"start","queue_duration_s":1.993335,"scheduling_latency_s":1.99337} gitlab # [ 167.298329] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.928Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8bcaf3cc264e7f3f7008e152","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8bcaf3cc264e7f3f7008e152: done: 0.019708 sec","job_status":"done","queue_duration_s":1.993335,"scheduling_latency_s":1.99337,"redis_calls":10,"redis_duration_s":0.002354,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000882,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001472,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019708,"completed_at":"2026-09-01T08:48:08.928Z","load_balancing_strategy":"primary","db_duration_s":0.003734,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.299896] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"57705faa0b54691d110c5944","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-57705faa0b54691d110c5944: start","job_status":"start","queue_duration_s":2.015005,"scheduling_latency_s":2.015031} gitlab # [ 167.309928] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"57705faa0b54691d110c5944","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-57705faa0b54691d110c5944: done: 0.018374 sec","job_status":"done","queue_duration_s":2.015005,"scheduling_latency_s":2.015031,"redis_calls":10,"redis_duration_s":0.002243,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001378,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018374,"completed_at":"2026-09-01T08:48:08.948Z","load_balancing_strategy":"primary","db_duration_s":0.00168,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.363404] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3f05f39f983cc5c7363ba0ce","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3f05f39f983cc5c7363ba0ce: start","job_status":"start","queue_duration_s":2.034806,"scheduling_latency_s":2.03483} gitlab # [ 167.368187] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3f05f39f983cc5c7363ba0ce","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3f05f39f983cc5c7363ba0ce: done: 0.018566 sec","job_status":"done","queue_duration_s":2.034806,"scheduling_latency_s":2.03483,"redis_calls":10,"redis_duration_s":0.0023049999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001436,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018566,"completed_at":"2026-09-01T08:48:08.968Z","load_balancing_strategy":"primary","db_duration_s":0.00186,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.428086] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0c1d41f631bac6f35f43d70c","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0c1d41f631bac6f35f43d70c: start","job_status":"start","queue_duration_s":2.054904,"scheduling_latency_s":2.054928} gitlab # [ 167.429198] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0c1d41f631bac6f35f43d70c","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0c1d41f631bac6f35f43d70c: done: 0.019607 sec","job_status":"done","queue_duration_s":2.054904,"scheduling_latency_s":2.054928,"redis_calls":10,"redis_duration_s":0.002213,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001263,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011404,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019607,"completed_at":"2026-09-01T08:48:08.989Z","load_balancing_strategy":"primary","db_duration_s":0.004132,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.511418] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:08.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e95caef459162f113f0cac1a","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e95caef459162f113f0cac1a: start","job_status":"start","queue_duration_s":2.076429,"scheduling_latency_s":2.076454} gitlab # [ 167.512422] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e95caef459162f113f0cac1a","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e95caef459162f113f0cac1a: done: 0.022223 sec","job_status":"done","queue_duration_s":2.076429,"scheduling_latency_s":2.076454,"redis_calls":10,"redis_duration_s":0.002278,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001366,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022223,"completed_at":"2026-09-01T08:48:09.013Z","load_balancing_strategy":"primary","db_duration_s":0.006137,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.588346] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5ec9678e7f35ce133cdb03fe","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5ec9678e7f35ce133cdb03fe: start","job_status":"start","queue_duration_s":2.100091,"scheduling_latency_s":2.100115} gitlab # [ 167.589408] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5ec9678e7f35ce133cdb03fe","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5ec9678e7f35ce133cdb03fe: done: 0.018282 sec","job_status":"done","queue_duration_s":2.100091,"scheduling_latency_s":2.100115,"redis_calls":10,"redis_duration_s":0.002337,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001418,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018282,"completed_at":"2026-09-01T08:48:09.033Z","load_balancing_strategy":"primary","db_duration_s":0.001835,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.652841] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f919c7bb85a3d1d7b472b805","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f919c7bb85a3d1d7b472b805: start","job_status":"start","queue_duration_s":2.119945,"scheduling_latency_s":2.11997} gitlab # [ 167.654046] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"f919c7bb85a3d1d7b472b805","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-f919c7bb85a3d1d7b472b805: done: 0.018829 sec","job_status":"done","queue_duration_s":2.119945,"scheduling_latency_s":2.11997,"redis_calls":10,"redis_duration_s":0.002202,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001315,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018829,"completed_at":"2026-09-01T08:48:09.054Z","load_balancing_strategy":"primary","db_duration_s":0.001792,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.666132] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bccc3fcde58eee43f0133df3","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bccc3fcde58eee43f0133df3: start","job_status":"start","queue_duration_s":2.140287,"scheduling_latency_s":2.140312} gitlab # [ 167.735402] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.075Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"bccc3fcde58eee43f0133df3","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-bccc3fcde58eee43f0133df3: done: 0.019837 sec","job_status":"done","queue_duration_s":2.140287,"scheduling_latency_s":2.140312,"redis_calls":10,"redis_duration_s":0.002257,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001356,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019837,"completed_at":"2026-09-01T08:48:09.075Z","load_balancing_strategy":"primary","db_duration_s":0.003727,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.736962] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"05575ab94118442c951b3306","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-05575ab94118442c951b3306: start","job_status":"start","queue_duration_s":2.161643,"scheduling_latency_s":2.161668} gitlab # [ 167.752417] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"05575ab94118442c951b3306","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-05575ab94118442c951b3306: done: 0.018396 sec","job_status":"done","queue_duration_s":2.161643,"scheduling_latency_s":2.161668,"redis_calls":10,"redis_duration_s":0.002325,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001415,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018396,"completed_at":"2026-09-01T08:48:09.095Z","load_balancing_strategy":"primary","db_duration_s":0.001806,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.804809] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e83deef1502b48d67dca2008","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e83deef1502b48d67dca2008: start","job_status":"start","queue_duration_s":2.181956,"scheduling_latency_s":2.181982} gitlab # [ 167.810592] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"e83deef1502b48d67dca2008","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-e83deef1502b48d67dca2008: done: 0.01847 sec","job_status":"done","queue_duration_s":2.181956,"scheduling_latency_s":2.181982,"redis_calls":10,"redis_duration_s":0.002281,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001412,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01847,"completed_at":"2026-09-01T08:48:09.115Z","load_balancing_strategy":"primary","db_duration_s":0.00171,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.878931] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"44559716e5409530e8cef44c","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-44559716e5409530e8cef44c: start","job_status":"start","queue_duration_s":2.201868,"scheduling_latency_s":2.201893} gitlab # [ 167.879865] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"44559716e5409530e8cef44c","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-44559716e5409530e8cef44c: done: 0.018591 sec","job_status":"done","queue_duration_s":2.201868,"scheduling_latency_s":2.201893,"redis_calls":10,"redis_duration_s":0.002453,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001487,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018591,"completed_at":"2026-09-01T08:48:09.135Z","load_balancing_strategy":"primary","db_duration_s":0.001885,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.941872] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d72172b2e64f69fb85a712f7","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d72172b2e64f69fb85a712f7: start","job_status":"start","queue_duration_s":2.221934,"scheduling_latency_s":2.221959} gitlab # [ 167.944246] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d72172b2e64f69fb85a712f7","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d72172b2e64f69fb85a712f7: done: 0.018974 sec","job_status":"done","queue_duration_s":2.221934,"scheduling_latency_s":2.221959,"redis_calls":10,"redis_duration_s":0.002207,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001293,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018974,"completed_at":"2026-09-01T08:48:09.156Z","load_balancing_strategy":"primary","db_duration_s":0.001859,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.994885] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"955eea26b0dbf5ce038c89ce","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-955eea26b0dbf5ce038c89ce: start","job_status":"start","queue_duration_s":2.242412,"scheduling_latency_s":2.242437} gitlab # [ 168.045931] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.179Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"955eea26b0dbf5ce038c89ce","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-955eea26b0dbf5ce038c89ce: done: 0.021415 sec","job_status":"done","queue_duration_s":2.242412,"scheduling_latency_s":2.242437,"redis_calls":10,"redis_duration_s":0.006411,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.005476,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021415,"completed_at":"2026-09-01T08:48:09.179Z","load_balancing_strategy":"primary","db_duration_s":0.001864,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.047281] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.180Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8895a1bd40469d6357aed38c","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8895a1bd40469d6357aed38c: start","job_status":"start","queue_duration_s":2.265223,"scheduling_latency_s":2.265243} gitlab # [ 168.107465] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"8895a1bd40469d6357aed38c","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-8895a1bd40469d6357aed38c: done: 0.021778 sec","job_status":"done","queue_duration_s":2.265223,"scheduling_latency_s":2.265243,"redis_calls":10,"redis_duration_s":0.002073,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001269,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012238,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021778,"completed_at":"2026-09-01T08:48:09.202Z","load_balancing_strategy":"primary","db_duration_s":0.007449,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.108648] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1c6c3f8bf4019c49eae4e377","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1c6c3f8bf4019c49eae4e377: start","job_status":"start","queue_duration_s":2.288492,"scheduling_latency_s":2.288516} gitlab # [ 168.162918] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1c6c3f8bf4019c49eae4e377","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1c6c3f8bf4019c49eae4e377: done: 0.018315 sec","job_status":"done","queue_duration_s":2.288492,"scheduling_latency_s":2.288516,"redis_calls":10,"redis_duration_s":0.002123,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001283,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018315,"completed_at":"2026-09-01T08:48:09.222Z","load_balancing_strategy":"primary","db_duration_s":0.001933,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.164765] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"216e6147a65a7ddf5fe7d762","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-216e6147a65a7ddf5fe7d762: start","job_status":"start","queue_duration_s":2.308102,"scheduling_latency_s":2.308161} gitlab # [ 168.220482] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"216e6147a65a7ddf5fe7d762","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-216e6147a65a7ddf5fe7d762: done: 0.017795 sec","job_status":"done","queue_duration_s":2.308102,"scheduling_latency_s":2.308161,"redis_calls":10,"redis_duration_s":0.002025,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001229,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017795,"completed_at":"2026-09-01T08:48:09.241Z","load_balancing_strategy":"primary","db_duration_s":0.001826,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.222418] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9fcdb844c5b8e878d478056c","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9fcdb844c5b8e878d478056c: start","job_status":"start","queue_duration_s":2.327356,"scheduling_latency_s":2.327385} gitlab # [ 168.237175] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"9fcdb844c5b8e878d478056c","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-9fcdb844c5b8e878d478056c: done: 0.020927 sec","job_status":"done","queue_duration_s":2.327356,"scheduling_latency_s":2.327385,"redis_calls":10,"redis_duration_s":0.0022849999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001406,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012516,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020927,"completed_at":"2026-09-01T08:48:09.263Z","load_balancing_strategy":"primary","db_duration_s":0.004175,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.317365] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7783f13843ebfd025b18a063","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7783f13843ebfd025b18a063: start","job_status":"start","queue_duration_s":2.350072,"scheduling_latency_s":2.350095} gitlab # [ 168.318827] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7783f13843ebfd025b18a063","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7783f13843ebfd025b18a063: done: 0.018795 sec","job_status":"done","queue_duration_s":2.350072,"scheduling_latency_s":2.350095,"redis_calls":10,"redis_duration_s":0.002162,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000773,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001389,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018795,"completed_at":"2026-09-01T08:48:09.284Z","load_balancing_strategy":"primary","db_duration_s":0.003564,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.422902] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"73a878a507c134bdf6ee4ddf","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-73a878a507c134bdf6ee4ddf: start","job_status":"start","queue_duration_s":2.370312,"scheduling_latency_s":2.370343} gitlab # [ 168.423883] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"73a878a507c134bdf6ee4ddf","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-73a878a507c134bdf6ee4ddf: done: 0.016789 sec","job_status":"done","queue_duration_s":2.370312,"scheduling_latency_s":2.370343,"redis_calls":10,"redis_duration_s":0.002132,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001302,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011461,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016789,"completed_at":"2026-09-01T08:48:09.302Z","load_balancing_strategy":"primary","db_duration_s":0.001408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.434499] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"03a6471859446cbbee3b4ddc","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-03a6471859446cbbee3b4ddc: start","job_status":"start","queue_duration_s":2.388545,"scheduling_latency_s":2.388566} gitlab # [ 168.449155] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"03a6471859446cbbee3b4ddc","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-03a6471859446cbbee3b4ddc: done: 0.01702 sec","job_status":"done","queue_duration_s":2.388545,"scheduling_latency_s":2.388566,"redis_calls":10,"redis_duration_s":0.00215,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000862,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001288,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011532,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01702,"completed_at":"2026-09-01T08:48:09.320Z","load_balancing_strategy":"primary","db_duration_s":0.001415,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.513881] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2e8eac5a4739fc1999b1970f","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2e8eac5a4739fc1999b1970f: start","job_status":"start","queue_duration_s":2.406943,"scheduling_latency_s":2.406965} gitlab # [ 168.514876] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2e8eac5a4739fc1999b1970f","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2e8eac5a4739fc1999b1970f: done: 0.018352 sec","job_status":"done","queue_duration_s":2.406943,"scheduling_latency_s":2.406965,"redis_calls":10,"redis_duration_s":0.002236,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001333,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018352,"completed_at":"2026-09-01T08:48:09.340Z","load_balancing_strategy":"primary","db_duration_s":0.001819,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.564858] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"dbeb9a2cc6883ef4945fa633","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-dbeb9a2cc6883ef4945fa633: start","job_status":"start","queue_duration_s":2.426607,"scheduling_latency_s":2.42663} gitlab # [ 168.565852] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.357Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"dbeb9a2cc6883ef4945fa633","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-dbeb9a2cc6883ef4945fa633: done: 0.015945 sec","job_status":"done","queue_duration_s":2.426607,"scheduling_latency_s":2.42663,"redis_calls":10,"redis_duration_s":0.0017670000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001087,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015945,"completed_at":"2026-09-01T08:48:09.357Z","load_balancing_strategy":"primary","db_duration_s":0.001491,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.642056] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4fb940946f4fed2ac2a237b3","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4fb940946f4fed2ac2a237b3: start","job_status":"start","queue_duration_s":2.443779,"scheduling_latency_s":2.443802} gitlab # [ 168.643066] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4fb940946f4fed2ac2a237b3","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4fb940946f4fed2ac2a237b3: done: 0.01873 sec","job_status":"done","queue_duration_s":2.443779,"scheduling_latency_s":2.443802,"redis_calls":10,"redis_duration_s":0.002111,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000785,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001326,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01873,"completed_at":"2026-09-01T08:48:09.377Z","load_balancing_strategy":"primary","db_duration_s":0.002034,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.701703] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"99485a2c16b9562ff559665f","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-99485a2c16b9562ff559665f: start","job_status":"start","queue_duration_s":2.464068,"scheduling_latency_s":2.464091} gitlab # [ 168.702954] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"99485a2c16b9562ff559665f","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-99485a2c16b9562ff559665f: done: 0.019244 sec","job_status":"done","queue_duration_s":2.464068,"scheduling_latency_s":2.464091,"redis_calls":10,"redis_duration_s":0.002003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000815,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001188,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019244,"completed_at":"2026-09-01T08:48:09.398Z","load_balancing_strategy":"primary","db_duration_s":0.003265,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.758519] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d2aef660834eb54d13985ea2","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d2aef660834eb54d13985ea2: start","job_status":"start","queue_duration_s":2.484624,"scheduling_latency_s":2.484648} gitlab # [ 168.759450] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"d2aef660834eb54d13985ea2","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-d2aef660834eb54d13985ea2: done: 0.015996 sec","job_status":"done","queue_duration_s":2.484624,"scheduling_latency_s":2.484648,"redis_calls":10,"redis_duration_s":0.0021000000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001119,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015996,"completed_at":"2026-09-01T08:48:09.415Z","load_balancing_strategy":"primary","db_duration_s":0.001295,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.817952] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5c5398e3f5eb00d767bf9096","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5c5398e3f5eb00d767bf9096: start","job_status":"start","queue_duration_s":2.501988,"scheduling_latency_s":2.50201} gitlab # [ 168.818987] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5c5398e3f5eb00d767bf9096","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5c5398e3f5eb00d767bf9096: done: 0.016367 sec","job_status":"done","queue_duration_s":2.501988,"scheduling_latency_s":2.50201,"redis_calls":10,"redis_duration_s":0.0019100000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000756,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001154,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016367,"completed_at":"2026-09-01T08:48:09.433Z","load_balancing_strategy":"primary","db_duration_s":0.001328,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.873089] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.435Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ff1531537fa0c7460bc93141","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ff1531537fa0c7460bc93141: start","job_status":"start","queue_duration_s":2.519915,"scheduling_latency_s":2.519937} gitlab # [ 168.874106] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.454Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ff1531537fa0c7460bc93141","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ff1531537fa0c7460bc93141: done: 0.019522 sec","job_status":"done","queue_duration_s":2.519915,"scheduling_latency_s":2.519937,"redis_calls":10,"redis_duration_s":0.002169,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000862,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001307,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019522,"completed_at":"2026-09-01T08:48:09.454Z","load_balancing_strategy":"primary","db_duration_s":0.002137,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.943939] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1acff046073abd7916a02183","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1acff046073abd7916a02183: start","job_status":"start","queue_duration_s":2.540793,"scheduling_latency_s":2.540817} gitlab # [ 168.945119] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"1acff046073abd7916a02183","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-1acff046073abd7916a02183: done: 0.021815 sec","job_status":"done","queue_duration_s":2.540793,"scheduling_latency_s":2.540817,"redis_calls":10,"redis_duration_s":0.002281,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001416,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021815,"completed_at":"2026-09-01T08:48:09.477Z","load_balancing_strategy":"primary","db_duration_s":0.005322,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.953866] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c94b9e224b424873f278a180","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c94b9e224b424873f278a180: start","job_status":"start","queue_duration_s":2.564023,"scheduling_latency_s":2.564045} gitlab # [ 168.968271] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.495Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"c94b9e224b424873f278a180","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-c94b9e224b424873f278a180: done: 0.016129 sec","job_status":"done","queue_duration_s":2.564023,"scheduling_latency_s":2.564045,"redis_calls":10,"redis_duration_s":0.002109,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000839,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00127,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016129,"completed_at":"2026-09-01T08:48:09.495Z","load_balancing_strategy":"primary","db_duration_s":0.001466,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.014041] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7a0e6555c773121d732ef2be","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7a0e6555c773121d732ef2be: start","job_status":"start","queue_duration_s":2.58179,"scheduling_latency_s":2.581815} gitlab # [ 169.028319] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7a0e6555c773121d732ef2be","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7a0e6555c773121d732ef2be: done: 0.025843 sec","job_status":"done","queue_duration_s":2.58179,"scheduling_latency_s":2.581815,"redis_calls":10,"redis_duration_s":0.005771,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000975,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004796,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025843,"completed_at":"2026-09-01T08:48:09.522Z","load_balancing_strategy":"primary","db_duration_s":0.006394,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.118950] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cb8969a77e6ef383b96690f0","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cb8969a77e6ef383b96690f0: start","job_status":"start","queue_duration_s":2.609353,"scheduling_latency_s":2.609384} gitlab # [ 169.119917] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"cb8969a77e6ef383b96690f0","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-cb8969a77e6ef383b96690f0: done: 0.020427 sec","job_status":"done","queue_duration_s":2.609353,"scheduling_latency_s":2.609384,"redis_calls":10,"redis_duration_s":0.002673,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000994,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001679,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020427,"completed_at":"2026-09-01T08:48:09.544Z","load_balancing_strategy":"primary","db_duration_s":0.001777,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.131872] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4aa8f723e858e73ba9192fea","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4aa8f723e858e73ba9192fea: start","job_status":"start","queue_duration_s":2.631167,"scheduling_latency_s":2.631191} gitlab # [ 169.192721] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4aa8f723e858e73ba9192fea","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4aa8f723e858e73ba9192fea: done: 0.017006 sec","job_status":"done","queue_duration_s":2.631167,"scheduling_latency_s":2.631191,"redis_calls":10,"redis_duration_s":0.002109,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00079,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001319,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017006,"completed_at":"2026-09-01T08:48:09.563Z","load_balancing_strategy":"primary","db_duration_s":0.001438,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.194379] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"32669ff20a5cc2f31e25d2d3","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-32669ff20a5cc2f31e25d2d3: start","job_status":"start","queue_duration_s":2.649813,"scheduling_latency_s":2.649838} gitlab # [ 169.248341] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"32669ff20a5cc2f31e25d2d3","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-32669ff20a5cc2f31e25d2d3: done: 0.017021 sec","job_status":"done","queue_duration_s":2.649813,"scheduling_latency_s":2.649838,"redis_calls":10,"redis_duration_s":0.0021680000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000769,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001399,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017021,"completed_at":"2026-09-01T08:48:09.582Z","load_balancing_strategy":"primary","db_duration_s":0.0016,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.249932] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"40ce1fabd27dc1290dff8310","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-40ce1fabd27dc1290dff8310: start","job_status":"start","queue_duration_s":2.668173,"scheduling_latency_s":2.668197} gitlab # [ 169.265165] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"40ce1fabd27dc1290dff8310","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-40ce1fabd27dc1290dff8310: done: 0.016376 sec","job_status":"done","queue_duration_s":2.668173,"scheduling_latency_s":2.668197,"redis_calls":10,"redis_duration_s":0.00199,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000782,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001208,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016376,"completed_at":"2026-09-01T08:48:09.599Z","load_balancing_strategy":"primary","db_duration_s":0.001499,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.329512] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7b7dc3a471fcdbc4b96f3c7c","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7b7dc3a471fcdbc4b96f3c7c: start","job_status":"start","queue_duration_s":2.685856,"scheduling_latency_s":2.685879} gitlab # [ 169.330573] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.622Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"7b7dc3a471fcdbc4b96f3c7c","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-7b7dc3a471fcdbc4b96f3c7c: done: 0.021518 sec","job_status":"done","queue_duration_s":2.685856,"scheduling_latency_s":2.685879,"redis_calls":10,"redis_duration_s":0.0024980000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001156,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001342,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021518,"completed_at":"2026-09-01T08:48:09.622Z","load_balancing_strategy":"primary","db_duration_s":0.001822,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.399675] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"235f37787431f55ed6e49d86","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-235f37787431f55ed6e49d86: start","job_status":"start","queue_duration_s":2.709837,"scheduling_latency_s":2.70987} gitlab # [ 169.400850] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"235f37787431f55ed6e49d86","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-235f37787431f55ed6e49d86: done: 0.02496 sec","job_status":"done","queue_duration_s":2.709837,"scheduling_latency_s":2.70987,"redis_calls":10,"redis_duration_s":0.002863,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001073,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00179,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02496,"completed_at":"2026-09-01T08:48:09.649Z","load_balancing_strategy":"primary","db_duration_s":0.002197,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.421582] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3ee40284b2adcea4c3c617bf","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3ee40284b2adcea4c3c617bf: start","job_status":"start","queue_duration_s":2.736674,"scheduling_latency_s":2.736708} gitlab # [ 169.483877] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3ee40284b2adcea4c3c617bf","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3ee40284b2adcea4c3c617bf: done: 0.025804 sec","job_status":"done","queue_duration_s":2.736674,"scheduling_latency_s":2.736708,"redis_calls":10,"redis_duration_s":0.002914,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001164,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00175,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025804,"completed_at":"2026-09-01T08:48:09.677Z","load_balancing_strategy":"primary","db_duration_s":0.00216,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.485600] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ad8eb1466db322a5a5c85838","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ad8eb1466db322a5a5c85838: start","job_status":"start","queue_duration_s":2.76411,"scheduling_latency_s":2.764166} gitlab # [ 169.538463] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ad8eb1466db322a5a5c85838","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ad8eb1466db322a5a5c85838: done: 0.020917 sec","job_status":"done","queue_duration_s":2.76411,"scheduling_latency_s":2.764166,"redis_calls":10,"redis_duration_s":0.002815,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001211,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001604,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020917,"completed_at":"2026-09-01T08:48:09.700Z","load_balancing_strategy":"primary","db_duration_s":0.001848,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.541340] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"17800b5cc8789b884d574758","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-17800b5cc8789b884d574758: start","job_status":"start","queue_duration_s":2.78676,"scheduling_latency_s":2.786787} gitlab # [ 169.615937] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"17800b5cc8789b884d574758","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-17800b5cc8789b884d574758: done: 0.019818 sec","job_status":"done","queue_duration_s":2.78676,"scheduling_latency_s":2.786787,"redis_calls":10,"redis_duration_s":0.002501,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001525,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019818,"completed_at":"2026-09-01T08:48:09.721Z","load_balancing_strategy":"primary","db_duration_s":0.002063,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.617254] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3bf9faf452c44fc7b2645905","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3bf9faf452c44fc7b2645905: start","job_status":"start","queue_duration_s":2.807998,"scheduling_latency_s":2.808021} gitlab # [ 169.626697] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"3bf9faf452c44fc7b2645905","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-3bf9faf452c44fc7b2645905: done: 0.023424 sec","job_status":"done","queue_duration_s":2.807998,"scheduling_latency_s":2.808021,"redis_calls":10,"redis_duration_s":0.002724,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001636,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023424,"completed_at":"2026-09-01T08:48:09.746Z","load_balancing_strategy":"primary","db_duration_s":0.00772,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.710429] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"22a0452cb736b5ab9deb266d","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-22a0452cb736b5ab9deb266d: start","job_status":"start","queue_duration_s":2.833352,"scheduling_latency_s":2.83338} gitlab # [ 169.711769] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:09.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"22a0452cb736b5ab9deb266d","created_at":"2026-09-01T08:48:06.746Z","trace_propagation_headers":{"sentry-trace":"c5edcb5d1f1f44edba27b4b9eb68068f-f1882399f3b04300","baggage":"sentry-trace_id=c5edcb5d1f1f44edba27b4b9eb68068f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"e9e1dccad299d6ca2749babb894d5a05","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:06.915Z","job_size_bytes":2,"pid":1700,"sidekiq_tid":"mak0","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-22a0452cb736b5ab9deb266d: done: 0.016982 sec","job_status":"done","queue_duration_s":2.833352,"scheduling_latency_s":2.83338,"redis_calls":10,"redis_duration_s":0.0018379999999999998,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001045,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016982,"completed_at":"2026-09-01T08:48:09.765Z","load_balancing_strategy":"primary","db_duration_s":0.001403,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 170.016242] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 5547500494 wd_nsec: 5547498439 gitlab # [ 179.241981] setup-gitlab-start[2373]: W, [2026-09-01T08:48:23.184956 #2373] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 204.350228] postgres[2610]: [2610] ERROR: relation "table_without_models" does not exist gitlab # [ 204.351177] postgres[2610]: [2610] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 207.905126] runuser[2372]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 207.907635] setup-gitlab-start[2349]: + systemctl restart gitlab-sidekiq.service gitlab # [ 207.922762] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 207.928949] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:51.871Z","message":"Shutting down"} gitlab # [ 207.929930] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:51.872Z","message":"Scheduler exiting..."} gitlab # [ 207.930837] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:51.872Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 207.931974] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:51.872Z","message":"Scheduler exiting..."} gitlab # [ 208.430291] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:52.373Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 211.432791] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:55.375Z","message":"Bye!"} gitlab # [ 211.447603] sidekiq-cluster[1700]: {"severity":"INFO","time":"2026-09-01T08:48:55.390Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1700,"worker_id":"sidekiq_0","memwd_rss_bytes":943161344,"retry":0} gitlab # [ 212.529945] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 212.531489] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 212.532578] systemd[1]: gitlab-sidekiq.service: Consumed 1min 10.538s CPU time over 2min 13.129s wall clock time, 935.6M memory peak, 4M read from disk. gitlab # [ 212.535474] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 212.536890] setup-gitlab-start[2349]: + mkdir -p /var/lib/secrets gitlab # [ 212.538186] setup-gitlab-start[2349]: + echo nixbot-test-token-of-20-chars gitlab # [ 212.539826] setup-gitlab-start[2349]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 212.541510] setup-gitlab-start[2349]: + curl -fs -X POST -H 'PRIVATE-TOKEN: nixbot-test-token-of-20-chars' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects gitlab # [ 213.286245] gitaly[912]: time="2026-09-01T08:48:57.229Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T08:49:07.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.228" pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.299848] gitaly[912]: time="2026-09-01T08:48:57.229Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T08:49:07.008" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.228" grpc.time_ms=0.266 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.412607] gitaly[912]: time="2026-09-01T08:48:57.355Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T08:49:07.135" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.355" pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.427840] gitaly[912]: time="2026-09-01T08:48:57.356Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T08:49:07.135" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.355" grpc.time_ms=0.243 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.477800] gitaly[912]: time="2026-09-01T08:48:57.421Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH diskcache=59092301-a561-4667-abad-c1423e662fcc grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-01T08:49:07.174" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.394" pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.491140] gitaly[912]: time="2026-09-01T08:48:57.421Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1571 command.oublock=32 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-01T08:49:07.174" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.394" grpc.time_ms=26.597 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.511353] gitaly[912]: time="2026-09-01T08:48:57.431Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T08:49:07.210" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.431" pid=912 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 213.527410] gitaly[912]: time="2026-09-01T08:48:57.431Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T08:49:07.210" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.431" grpc.time_ms=0.414 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.528473] gitaly[912]: time="2026-09-01T08:48:57.442Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T08:49:07.217" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.438" grpc.time_ms=4.608 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.573252] gitaly[912]: time="2026-09-01T08:48:57.516Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T08:49:07.295" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.516" pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.574266] gitaly[912]: time="2026-09-01T08:48:57.516Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T08:49:07.295" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.516" grpc.time_ms=0.238 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.623610] gitaly[912]: time="2026-09-01T08:48:57.566Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH diskcache=6d65026d-78c5-433c-832a-73bf3e5bb04f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-01T08:49:07.004" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.545" pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.630339] gitaly[912]: time="2026-09-01T08:48:57.573Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T08:49:07.032" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.573" pid=912 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 213.632275] gitaly[912]: time="2026-09-01T08:48:57.573Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T08:49:07.032" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.573" grpc.time_ms=0.403 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.640437] gitaly[912]: time="2026-09-01T08:48:57.583Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T08:49:07.038" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.579" grpc.time_ms=4.418 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.666419] gitaly[912]: time="2026-09-01T08:48:57.609Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T08:49:07.069" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.609" grpc.time_ms=0.186 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.673776] gitaly[912]: time="2026-09-01T08:48:57.617Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-01T08:49:07.076" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.616" grpc.time_ms=0.141 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.709137] gitaly[912]: time="2026-09-01T08:48:57.652Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1581 command.oublock=32 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-01T08:49:07.004" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:57.545" grpc.time_ms=21.75 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.262268] gitaly[912]: time="2026-09-01T08:48:58.205Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GFNA4W7PDP1RD202TQDH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T08:49:08.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:58.200" grpc.time_ms=4.741 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 214.579849] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:48:58 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 2031 gitlab # [ 214.582623] setup-gitlab-start[2349]: + mkdir -p /tmp/test-flake gitlab # [ 214.583816] setup-gitlab-start[2349]: + cd /tmp/test-flake gitlab # [ 214.584874] setup-gitlab-start[2349]: + git init -b master gitlab # [ 214.594595] setup-gitlab-start[2704]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 214.596448] setup-gitlab-start[2349]: + git config user.name test gitlab # [ 214.597768] setup-gitlab-start[2349]: + git config user.email test@example.com gitlab # [ 214.599339] setup-gitlab-start[2349]: + cat gitlab # [ 214.600345] setup-gitlab-start[2349]: + git add flake.nix gitlab # [ 214.601421] setup-gitlab-start[2349]: + git commit -m 'initial commit' gitlab # [ 214.605349] setup-gitlab-start[2709]: [master (root-commit) 4360d82] initial commit gitlab # [ 214.606856] setup-gitlab-start[2709]: 1 file changed, 10 insertions(+) gitlab # [ 214.608153] setup-gitlab-start[2709]: create mode 100644 flake.nix gitlab # [ 214.609415] setup-gitlab-start[2349]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 214.611479] setup-gitlab-start[2349]: + git push -u origin master gitlab # [ 214.656906] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:48:58 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 32 gitlab # [ 214.751663] gitaly[912]: time="2026-09-01T08:48:58.694Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHQBV0K7DE9M7JWJH8QK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:48:58.688" grpc.time_ms=5.83 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 214.754037] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:48:58 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 93 gitlab # [ 215.529495] gitaly[912]: time="2026-09-01T08:48:59.472Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T08:49:09.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:59.467" grpc.time_ms=5.36 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.580741] gitaly[912]: time="2026-09-01T08:48:59.523Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=324 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-01T08:49:29.017" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-01T08:48:59.518" grpc.time_ms=5.415 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.622519] gitaly[912]: time="2026-09-01T08:48:59.565Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG duration_ms=725ns method=POST pid=912 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 215.653088] gitaly[912]: time="2026-09-01T08:48:59.596Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG duration_ms=29ns method=POST pid=912 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 215.657440] gitaly[912]: time="2026-09-01T08:48:59.600Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5309 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:48:58.838" grpc.time_ms=758.036 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 215.693150] gitaly[912]: time="2026-09-01T08:48:59.636Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4779 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:48:59.635" grpc.time_ms=0.472 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 215.727871] gitaly[912]: time="2026-09-01T08:48:59.671Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4778 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:48:59.670" grpc.time_ms=0.55 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 215.762032] gitaly[912]: time="2026-09-01T08:48:59.705Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:48:59.704" grpc.time_ms=0.334 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 215.880715] gitaly[912]: time="2026-09-01T08:48:59.823Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T08:49:09.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:48:59.819" grpc.time_ms=4.777 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.914604] gitaly[912]: time="2026-09-01T08:48:59.857Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1559 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-01T08:49:09.037" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:48:59.848" grpc.time_ms=9.647 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 215.944044] gitaly[912]: time="2026-09-01T08:48:59.887Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG duration_ms=149ns method=POST pid=912 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 215.948560] gitaly[912]: time="2026-09-01T08:48:59.891Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4803 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:48:59.735" grpc.time_ms=155.744 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 215.968717] gitaly[912]: time="2026-09-01T08:48:59.911Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG diskcache=bd58f804-c42c-4ecd-86f7-670d679a0ad2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:48:58.796" pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 215.982527] gitaly[912]: time="2026-09-01T08:48:59.912Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=164 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=901120 command.minflt=13958 command.oublock=32 command.real_time_ms=1109 command.spawn_token_fork_ms=0 command.system_time_ms=92 command.user_time_ms=72 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:48:58.796" grpc.time_ms=1115.773 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 215.983962] gitlab-workhorse[822]: time="2026-09-01T08:48:59Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 215.984918] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:48:59 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 1192 gitlab # [ 216.009315] setup-gitlab-start[2713]: To http://gitlab/root/test-flake.git gitlab # [ 216.010433] setup-gitlab-start[2713]: * [new branch] master -> master gitlab # [ 216.012098] setup-gitlab-start[2713]: branch 'master' set up to track 'origin/master'. gitlab # [ 216.016252] systemd[1]: Finished setup-gitlab.service. gitlab # [ 216.023175] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 216.90 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 216.978288] sidekiq-cluster[2657]: {"severity":"INFO","time":"2026-09-01T08:49:00.899Z","message":"Starting cluster with 1 processes"} gitlab # [ 217.019928] nixbot[2804]: {"ts": "2026-09-01T08:49:00.962Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 217.037260] nixbot[2804]: {"ts": "2026-09-01T08:49:00.980Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 217.050109] nixbot[2804]: {"ts": "2026-09-01T08:49:00.993Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 217.052839] nixbot[2804]: {"ts": "2026-09-01T08:49:00.996Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 217.056265] nixbot[2804]: {"ts": "2026-09-01T08:49:00.999Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 217.061252] nixbot[2804]: {"ts": "2026-09-01T08:49:01.004Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 217.063675] nixbot[2804]: {"ts": "2026-09-01T08:49:01.006Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 217.065186] nixbot[2804]: {"ts": "2026-09-01T08:49:01.008Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 217.069242] nixbot[2804]: {"ts": "2026-09-01T08:49:01.012Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 217.071153] nixbot[2804]: {"ts": "2026-09-01T08:49:01.014Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 217.075498] nixbot[2804]: {"ts": "2026-09-01T08:49:01.018Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 217.080475] nixbot[2804]: {"ts": "2026-09-01T08:49:01.023Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 217.083541] nixbot[2804]: {"ts": "2026-09-01T08:49:01.026Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 217.085669] nixbot[2804]: {"ts": "2026-09-01T08:49:01.028Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 217.088881] nixbot[2804]: {"ts": "2026-09-01T08:49:01.032Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 217.090667] nixbot[2804]: {"ts": "2026-09-01T08:49:01.033Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 217.092584] nixbot[2804]: {"ts": "2026-09-01T08:49:01.035Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 217.094072] nixbot[2804]: {"ts": "2026-09-01T08:49:01.037Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 217.097407] nixbot[2804]: {"ts": "2026-09-01T08:49:01.040Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 217.099231] nixbot[2804]: {"ts": "2026-09-01T08:49:01.042Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 217.100879] nixbot[2804]: {"ts": "2026-09-01T08:49:01.044Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 217.104832] nixbot[2804]: {"ts": "2026-09-01T08:49:01.048Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 217.106476] nixbot[2804]: {"ts": "2026-09-01T08:49:01.049Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 217.108650] nixbot[2804]: {"ts": "2026-09-01T08:49:01.051Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 217.110462] nixbot[2804]: {"ts": "2026-09-01T08:49:01.053Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 217.112966] nixbot[2804]: {"ts": "2026-09-01T08:49:01.056Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 217.114871] nixbot[2804]: {"ts": "2026-09-01T08:49:01.058Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 217.416589] nixbot[2804]: INFO: Started server process [2804] gitlab # [ 217.417958] nixbot[2804]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 217.437432] nixbot[2804]: INFO: ::ffff:127.0.0.1:40486 - "GET /health HTTP/1.1" 200 OK gitlab # [ 217.441410] systemd[1]: Started nixbot CI service. gitlab # [ 217.443563] systemd[1]: Reached target Multi-User System. gitlab # [ 217.445866] systemd[1]: Startup finished in 1.108s (kernel) + 3.199s (initrd) + 3min 33.137s (userspace) = 3min 37.445s. gitlab # [ 217.650561] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:01 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 214 gitlab # [ 217.655457] nixbot[2804]: {"ts": "2026-09-01T08:49:01.598Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 217.672512] nixbot[2804]: {"ts": "2026-09-01T08:49:01.615Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 217.781973] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:01 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 98 gitlab # [ 217.785899] nixbot[2804]: {"ts": "2026-09-01T08:49:01.729Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 217.790048] nixbot[2804]: {"ts": "2026-09-01T08:49:01.733Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 217.894890] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:01 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 919 "" "python-httpx/0.28.1" 99 gitlab # [ 217.898584] nixbot[2804]: {"ts": "2026-09-01T08:49:01.838Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab: (finished: waiting for unit nixbot.service, in 2.04 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 218.175327] nixbot[2804]: INFO: ::ffff:127.0.0.1:40488 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.02 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 218.240512] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "curl/8.21.0" 50 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.07 seconds) [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-01T08:49:01.821Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] (finished: subtest: project discovered and webhook registered, in 0.07 seconds) subtest: push delivers the webhook gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 218.298123] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:02 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 27 gitlab # [ 218.377741] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:02 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 204 "" "git/2.55.0" 74 gitlab # [ 218.382950] gitaly[912]: time="2026-09-01T08:49:02.325Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GN95EZD03XMPJC891SF8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:02.310" grpc.time_ms=8.785 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 137.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 218.662367] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:02 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 759 gitlab # [ 218.666357] nixbot[2804]: {"ts": "2026-09-01T08:49:02.606Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 218.750457] gitaly[912]: time="2026-09-01T08:49:02.693Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=327 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-01T08:49:32.088" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-01T08:49:02.688" grpc.time_ms=5.065 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.776846] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "python-httpx/0.28.1" 105 gitlab # [ 218.779716] nixbot[2804]: {"ts": "2026-09-01T08:49:02.720Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 218.790410] gitaly[912]: time="2026-09-01T08:49:02.733Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V duration_ms=162ns method=POST pid=912 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 218.826430] gitaly[912]: time="2026-09-01T08:49:02.769Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V duration_ms=31ns method=POST pid=912 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 218.831078] gitaly[912]: time="2026-09-01T08:49:02.770Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5309 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:49:02.570" grpc.time_ms=199.363 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 218.844567] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:02 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 919 "" "python-httpx/0.28.1" 59 gitlab # [ 218.847395] nixbot[2804]: {"ts": "2026-09-01T08:49:02.787Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 218.876106] gitaly[912]: time="2026-09-01T08:49:02.818Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4779 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:49:02.818" grpc.time_ms=0.513 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 218.910415] gitaly[912]: time="2026-09-01T08:49:02.853Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4778 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:49:02.851" grpc.time_ms=2.211 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 218.925767] gitaly[912]: time="2026-09-01T08:49:02.861Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GNT8R4HFPQ422EYHXWHQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-01T08:49:32.254" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:49:02.854" grpc.time_ms=6.4 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.960821] gitaly[912]: time="2026-09-01T08:49:02.904Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GNT8R4HFPQ422EYHXWHQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:49:32.302" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:49:02.903" grpc.time_ms=0.626 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 218.978481] gitaly[912]: time="2026-09-01T08:49:02.911Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4780 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:49:02.911" grpc.time_ms=0.39 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 218.991067] gitaly[912]: time="2026-09-01T08:49:02.931Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GNT8R4HFPQ422EYHXWHQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-01T08:49:12.000" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:49:02.913" grpc.time_ms=12.522 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 219.015069] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 731 "" "python-httpx/0.28.1" 165 gitlab # [ 219.018607] nixbot[2804]: {"ts": "2026-09-01T08:49:02.959Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 219.036409] gitaly[912]: time="2026-09-01T08:49:02.979Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V duration_ms=34ns method=POST pid=912 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 219.040634] gitaly[912]: time="2026-09-01T08:49:02.980Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4803 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:49:02.944" grpc.time_ms=35.797 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 219.055878] gitaly[912]: time="2026-09-01T08:49:02.994Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V diskcache=6497d48a-d7c0-49d3-b02c-1e8a56a7dd25 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:02.529" pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 219.056770] gitlab-workhorse[822]: time="2026-09-01T08:49:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 219.057746] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:02 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 635 gitlab # [ 219.058842] gitaly[912]: time="2026-09-01T08:49:02.994Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=144 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=868352 command.minflt=13646 command.oublock=32 command.real_time_ms=459 command.spawn_token_fork_ms=4 command.system_time_ms=88 command.user_time_ms=56 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:02.529" grpc.time_ms=464.659 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # To http://gitlab/root/test-flake.git gitlab # 4360d82..ee9c51b master -> master gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.85 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 219.116327] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:03 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 96 gitlab # [ 219.160290] nixbot[2804]: {"ts": "2026-09-01T08:49:03.103Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 219.166270] nixbot[2804]: INFO: ::ffff:127.0.0.1:40490 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab # [ 219.169164] nixbot[2804]: {"ts": "2026-09-01T08:49:03.110Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.07 seconds) [] gitlab # [ 219.237913] gitaly[912]: time="2026-09-01T08:49:03.181Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GP4X0YRVR10T6ZMCF30A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:03.176" grpc.time_ms=4.946 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 219.259331] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 69 gitlab # [ 219.332147] gitaly[912]: time="2026-09-01T08:49:03.275Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GP76X0A7X2JDC5C1ND3C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:03.269" pid=912 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 219.333168] gitaly[912]: time="2026-09-01T08:49:03.275Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GP76X0A7X2JDC5C1ND3C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:03.269" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 219.359663] gitaly[912]: time="2026-09-01T08:49:03.302Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GP76X0A7X2JDC5C1ND3C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:03.269" grpc.time_ms=33.001 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 219.376818] gitlab-workhorse[822]: time="2026-09-01T08:49:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 219.380505] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 113 gitlab # [ 219.445808] gitaly[912]: time="2026-09-01T08:49:03.388Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPAVBDTWM84MPVARH0DV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:03.377" pid=912 remote_ip="2001:db8:1::1" request_sha=0edeb097aab505e1071ae26f550810024298eccc response_bytes=419 span.kind=server system=grpc gitlab # [ 219.446738] gitaly[912]: time="2026-09-01T08:49:03.388Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1198 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPAVBDTWM84MPVARH0DV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:03.377" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 219.447463] gitaly[912]: time="2026-09-01T08:49:03.389Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1198 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPAVBDTWM84MPVARH0DV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:03.377" grpc.time_ms=11.598 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 219.498784] gitlab-workhorse[822]: time="2026-09-01T08:49:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=419 gitlab # [ 219.499620] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 419 "" "git/2.55.0" 118 gitlab # [ 219.575930] gitaly[912]: time="2026-09-01T08:49:03.517Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GPFBXG04REZR7RMGBB0H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:03.510" grpc.time_ms=6.639 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 219.592911] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 68 gitlab # [ 219.647044] gitaly[912]: time="2026-09-01T08:49:03.590Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPHMJ7FQ9YPDZDJ2XVB3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:03.584" pid=912 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 219.648076] gitaly[912]: time="2026-09-01T08:49:03.590Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPHMJ7FQ9YPDZDJ2XVB3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:03.584" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 219.649130] gitaly[912]: time="2026-09-01T08:49:03.590Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPHMJ7FQ9YPDZDJ2XVB3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:03.584" grpc.time_ms=6.021 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 219.698785] gitlab-workhorse[822]: time="2026-09-01T08:49:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 219.699820] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 101 gitlab # [ 219.776092] gitaly[912]: time="2026-09-01T08:49:03.719Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GPNN5R1J24JGZXWQ77JH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:03.713" grpc.time_ms=5.654 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 219.777244] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:03 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 49 gitlab # [ 219.854751] gitaly[912]: time="2026-09-01T08:49:03.798Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPQW7GE71TETZZTNBQSS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:03.789" pid=912 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 219.855667] gitaly[912]: time="2026-09-01T08:49:03.798Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1165 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPQW7GE71TETZZTNBQSS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:03.789" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 219.856308] gitaly[912]: time="2026-09-01T08:49:03.798Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1165 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPQW7GE71TETZZTNBQSS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:03.789" grpc.time_ms=8.982 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 219.901949] gitlab-workhorse[822]: time="2026-09-01T08:49:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 219.902753] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 104 gitlab # [ 219.982390] gitaly[912]: time="2026-09-01T08:49:03.925Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:49:33.424" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:49:03.924" grpc.time_ms=0.563 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 220.029383] gitaly[912]: time="2026-09-01T08:49:03.972Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-01T08:49:33.467" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:49:03.967" grpc.time_ms=5.12 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 220.063654] gitaly[912]: time="2026-09-01T08:49:04.006Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:49:14.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:49:04.002" grpc.time_ms=4.285 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 220.100090] gitaly[912]: time="2026-09-01T08:49:04.043Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:49:14.036" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:49:04.037" grpc.time_ms=5.984 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 220.170917] gitaly[912]: time="2026-09-01T08:49:04.114Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-01T08:49:14.113" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:49:04.113" grpc.time_ms=0.182 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 220.191783] gitaly[912]: time="2026-09-01T08:49:04.134Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=620 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-01T08:49:14.128" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:49:04.128" grpc.time_ms=6.192 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 220.201563] nixbot[2804]: INFO: ::ffff:127.0.0.1:40500 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.04 seconds) [] gitlab # [ 220.241699] gitaly[912]: time="2026-09-01T08:49:04.184Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:49:04.183" grpc.time_ms=0.555 pid=912 span.kind=server system=grpc gitlab # [ 220.272333] gitaly[912]: time="2026-09-01T08:49:04.215Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:49:04.215" grpc.time_ms=0.364 pid=912 span.kind=server system=grpc gitlab # [ 220.290048] gitaly[912]: time="2026-09-01T08:49:04.232Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 diskcache=880f8984-c035-490f-981e-17a4b94d66a4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T08:49:14.156" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:49:04.156" pid=912 ref=refs/keep-around/4360d82929f184c6583c9f707539c5b803e775b0 remote_ip="2001:db8:1::1" revision=4360d82929f184c6583c9f707539c5b803e775b0 span.kind=server system=grpc username=root gitlab # [ 220.305994] gitaly[912]: time="2026-09-01T08:49:04.233Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=47 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5247 command.oublock=8 command.real_time_ms=71 command.spawn_token_fork_ms=0 command.system_time_ms=26 command.user_time_ms=21 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T08:49:14.156" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:49:04.156" grpc.time_ms=76.199 pid=912 ref=refs/keep-around/4360d82929f184c6583c9f707539c5b803e775b0 remote_ip="2001:db8:1::1" revision=4360d82929f184c6583c9f707539c5b803e775b0 span.kind=server system=grpc username=root gitlab # [ 220.324549] gitaly[912]: time="2026-09-01T08:49:04.236Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:49:34.035" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:49:04.235" grpc.time_ms=0.376 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 220.449097] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:04 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4360d82929f184c6583c9f707539c5b803e775b0 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 521 gitlab # [ 220.452845] nixbot[2804]: {"ts": "2026-09-01T08:49:04.393Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4360d82929f184c6583c9f707539c5b803e775b0 \"HTTP/1.1 201 Created\""} gitlab # [ 220.457307] nixbot[2804]: {"ts": "2026-09-01T08:49:04.400Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-4360d82929f1-0"} gitlab # [ 220.579543] nixbot[2804]: {"ts": "2026-09-01T08:49:04.522Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-4360d82929f1-0", "argv0": "bwrap"} gitlab # [ 220.656908] systemd[1]: Started Nix Daemon. gitlab # [ 220.699658] nix-daemon[3027]: accepted connection from pid 3022, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 221.226453] nixbot[2804]: INFO: ::ffff:127.0.0.1:40502 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 222.245285] nixbot[2804]: INFO: ::ffff:127.0.0.1:40504 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.01 seconds) [] gitlab # [ 223.263149] nixbot[2804]: INFO: ::ffff:127.0.0.1:59636 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 224.278931] nixbot[2804]: INFO: ::ffff:127.0.0.1:59650 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab # [ 224.917935] nix-daemon[3027]: accepted connection from pid 3050, user nixbot gitlab # [ 224.929078] nix-daemon[3027]: accepted connection from pid 3022, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 225.296530] nixbot[2804]: INFO: ::ffff:127.0.0.1:59662 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 226.314497] nixbot[2804]: INFO: ::ffff:127.0.0.1:59672 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 227.333126] nixbot[2804]: INFO: ::ffff:127.0.0.1:59688 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 228.350762] nixbot[2804]: INFO: ::ffff:127.0.0.1:59702 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 229.368749] nixbot[2804]: INFO: ::ffff:127.0.0.1:59712 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab # [ 229.826635] nix-daemon[3027]: accepted connection from pid 3022, user nixbot gitlab # [ 229.872506] nix-daemon[3027]: accepted connection from pid 3084, user nixbot gitlab # [ 229.908772] gitaly[912]: time="2026-09-01T08:49:13.851Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2H0JDNBYAR7DFE79W6TNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:49:43.250" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:49:13.851" grpc.time_ms=0.51 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 229.977135] gitaly[912]: time="2026-09-01T08:49:13.920Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2H0JDNBYAR7DFE79W6TNJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:49:23.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:49:13.915" grpc.time_ms=4.619 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 230.075326] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:14 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4360d82929f184c6583c9f707539c5b803e775b0 HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 209 gitlab # [ 230.076257] nixbot[2804]: {"ts": "2026-09-01T08:49:14.016Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4360d82929f184c6583c9f707539c5b803e775b0 \"HTTP/1.1 201 Created\""} gitlab # [ 230.124118] gitaly[912]: time="2026-09-01T08:49:14.067Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2H0S93RG73BV6CD4ZHQJ1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:49:44.066" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:49:14.066" grpc.time_ms=0.543 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 230.168234] gitaly[912]: time="2026-09-01T08:49:14.111Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2H0S93RG73BV6CD4ZHQJ1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:49:24.106" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:49:14.106" grpc.time_ms=4.827 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 230.229362] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:14 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4360d82929f184c6583c9f707539c5b803e775b0 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 146 gitlab # [ 230.230453] nixbot[2804]: {"ts": "2026-09-01T08:49:14.173Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4360d82929f184c6583c9f707539c5b803e775b0 \"HTTP/1.1 201 Created\""} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 230.387698] nixbot[2804]: INFO: ::ffff:127.0.0.1:59714 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 231.405352] nixbot[2804]: INFO: ::ffff:127.0.0.1:59718 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 232.423087] nixbot[2804]: INFO: ::ffff:127.0.0.1:59734 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab # [ 233.440679] nixbot[2804]: INFO: ::ffff:127.0.0.1:33084 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab # [ 234.027826] nix-daemon[3027]: accepted connection from pid 3125, user nixbot gitlab # [ 234.095335] gitaly[912]: time="2026-09-01T08:49:18.038Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2H4N6FGAJW90WRM3K55GP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:49:48.037" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:49:18.037" grpc.time_ms=0.496 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 234.124528] gitaly[912]: time="2026-09-01T08:49:18.067Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2H4N6FGAJW90WRM3K55GP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:49:28.062" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:49:18.062" grpc.time_ms=5.095 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 234.167309] nixbot[2804]: {"ts": "2026-09-01T08:49:18.110Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4360d82929f184c6583c9f707539c5b803e775b0 \"HTTP/1.1 201 Created\""} gitlab # [ 234.171562] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:18 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4360d82929f184c6583c9f707539c5b803e775b0 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 123 gitlab # [ 234.238845] gitaly[912]: time="2026-09-01T08:49:18.182Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2H4STFY9VGBF18CJWF8N4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:18.174" grpc.time_ms=6.912 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.240756] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:18 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 43 gitlab # [ 234.311860] gitaly[912]: time="2026-09-01T08:49:18.254Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2H4V9W8C662A50QJKFN1G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:18.248" pid=912 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 234.313127] gitaly[912]: time="2026-09-01T08:49:18.254Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2H4V9W8C662A50QJKFN1G grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:18.248" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 234.314547] gitlab-workhorse[822]: time="2026-09-01T08:49:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 234.315912] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:49:18 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 70 gitlab # [ 234.345128] gitaly[912]: time="2026-09-01T08:49:18.288Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2H4V9W8C662A50QJKFN1G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:49:18.248" grpc.time_ms=6.323 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 234.459356] nixbot[2804]: INFO: ::ffff:127.0.0.1:33088 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 235.478354] nixbot[2804]: INFO: ::ffff:127.0.0.1:33100 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 236.497914] nixbot[2804]: INFO: ::ffff:127.0.0.1:33102 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 237.519625] nixbot[2804]: INFO: ::ffff:127.0.0.1:33106 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 238.539912] nixbot[2804]: INFO: ::ffff:127.0.0.1:33114 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 239.559337] nixbot[2804]: INFO: ::ffff:127.0.0.1:33130 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 240.602534] nixbot[2804]: INFO: ::ffff:127.0.0.1:33138 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.05 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 241.629256] nixbot[2804]: INFO: ::ffff:127.0.0.1:33144 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 242.649070] nixbot[2804]: INFO: ::ffff:127.0.0.1:33146 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 243.670560] nixbot[2804]: INFO: ::ffff:127.0.0.1:42166 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 244.690338] nixbot[2804]: INFO: ::ffff:127.0.0.1:42168 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 245.708941] nixbot[2804]: INFO: ::ffff:127.0.0.1:42178 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 246.728564] nixbot[2804]: INFO: ::ffff:127.0.0.1:42180 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 247.748309] nixbot[2804]: INFO: ::ffff:127.0.0.1:42190 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 248.770416] nixbot[2804]: INFO: ::ffff:127.0.0.1:42196 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab # [ 249.294767] sidekiq-cluster[2816]: 2026-09-01T08:49:33.237Z pid=2816 tid=xig INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2816, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>439390208} gitlab # [ 249.546404] sidekiq-cluster[2816]: W, [2026-09-01T08:49:33.489448 #2816] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 249.605924] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:49:33.549Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 249.789694] nixbot[2804]: INFO: ::ffff:127.0.0.1:42200 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 250.807359] nixbot[2804]: INFO: ::ffff:127.0.0.1:42216 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab # [ 251.650618] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:49:35.593Z","message":"GitLab reliable fetch activated!"} gitlab # [ 251.651511] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:49:35.593Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 251.827699] nixbot[2804]: INFO: ::ffff:127.0.0.1:42226 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 252.847833] nixbot[2804]: INFO: ::ffff:127.0.0.1:42230 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 253.867698] nixbot[2804]: INFO: ::ffff:127.0.0.1:33812 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 254.888832] nixbot[2804]: INFO: ::ffff:127.0.0.1:33820 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 255.908087] nixbot[2804]: INFO: ::ffff:127.0.0.1:33824 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 256.925059] nixbot[2804]: INFO: ::ffff:127.0.0.1:33832 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 257.943745] nixbot[2804]: INFO: ::ffff:127.0.0.1:33848 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 258.962050] nixbot[2804]: INFO: ::ffff:127.0.0.1:33864 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 259.980986] nixbot[2804]: INFO: ::ffff:127.0.0.1:33866 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 261.000777] nixbot[2804]: INFO: ::ffff:127.0.0.1:33872 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 262.020168] nixbot[2804]: INFO: ::ffff:127.0.0.1:33880 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 263.040927] nixbot[2804]: INFO: ::ffff:127.0.0.1:32898 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 264.061298] nixbot[2804]: INFO: ::ffff:127.0.0.1:32914 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 265.081932] nixbot[2804]: INFO: ::ffff:127.0.0.1:32920 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 266.100169] nixbot[2804]: INFO: ::ffff:127.0.0.1:32934 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 267.119599] nixbot[2804]: INFO: ::ffff:127.0.0.1:32940 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 268.146478] nixbot[2804]: INFO: ::ffff:127.0.0.1:32954 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.03 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 269.166583] nixbot[2804]: INFO: ::ffff:127.0.0.1:32960 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 270.184502] nixbot[2804]: INFO: ::ffff:127.0.0.1:32972 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 271.202334] nixbot[2804]: INFO: ::ffff:127.0.0.1:32976 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 272.221629] nixbot[2804]: INFO: ::ffff:127.0.0.1:32992 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 273.241593] nixbot[2804]: INFO: ::ffff:127.0.0.1:44574 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 274.260034] nixbot[2804]: INFO: ::ffff:127.0.0.1:44580 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab # [ 275.175172] postgres[3307]: [3307] ERROR: relation "table_without_models" does not exist gitlab # [ 275.176050] postgres[3307]: [3307] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab # [ 275.281310] nixbot[2804]: INFO: ::ffff:127.0.0.1:44588 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab # [ 276.296784] nixbot[2804]: INFO: ::ffff:127.0.0.1:44602 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab # [ 277.312720] nixbot[2804]: INFO: ::ffff:127.0.0.1:44608 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab # [ 278.334759] nixbot[2804]: INFO: ::ffff:127.0.0.1:44624 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab # [ 278.377461] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:02.319Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 278.379921] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:02.320Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 278.382324] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:02.320Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 278.384448] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:02.320Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 278.386881] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:02.322Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 279.351898] nixbot[2804]: INFO: ::ffff:127.0.0.1:44638 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab # [ 279.864545] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:03.807Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"ffca85472e38f1614c8c0fec","created_at":"2026-09-01T08:48:57.464Z","trace_propagation_headers":{"sentry-trace":"f6a6604af6af4e88a20fee572ace0d33-a63abaaba65c4d67","baggage":"sentry-trace_id=f6a6604af6af4e88a20fee572ace0d33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GFNA4W7PDP1RD202TQDH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:57.494Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-ffca85472e38f1614c8c0fec: start","job_status":"start","queue_duration_s":66.312437,"scheduling_latency_s":66.312485} gitlab # [ 280.033257] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:03.975Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"ffca85472e38f1614c8c0fec","created_at":"2026-09-01T08:48:57.464Z","trace_propagation_headers":{"sentry-trace":"f6a6604af6af4e88a20fee572ace0d33-a63abaaba65c4d67","baggage":"sentry-trace_id=f6a6604af6af4e88a20fee572ace0d33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GFNA4W7PDP1RD202TQDH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:57.494Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-ffca85472e38f1614c8c0fec: done: 0.16893 sec","job_status":"done","queue_duration_s":66.312437,"scheduling_latency_s":66.312485,"redis_calls":23,"redis_duration_s":0.008744,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.005395,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001938,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001411,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.090061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.08793883,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.16893,"completed_at":"2026-09-01T08:50:03.975Z","load_balancing_strategy":"primary","db_duration_s":0.041859,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 280.097719] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:03.978Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f0392869101c3eff05ab568a","created_at":"2026-09-01T08:48:57.505Z","trace_propagation_headers":{"sentry-trace":"f6a6604af6af4e88a20fee572ace0d33-a63abaaba65c4d67","baggage":"sentry-trace_id=f6a6604af6af4e88a20fee572ace0d33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GFNA4W7PDP1RD202TQDH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:57.511Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f0392869101c3eff05ab568a: start","job_status":"start","queue_duration_s":66.466324,"scheduling_latency_s":66.466356} gitlab # [ 280.129299] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:04.071Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"f0392869101c3eff05ab568a","created_at":"2026-09-01T08:48:57.505Z","trace_propagation_headers":{"sentry-trace":"f6a6604af6af4e88a20fee572ace0d33-a63abaaba65c4d67","baggage":"sentry-trace_id=f6a6604af6af4e88a20fee572ace0d33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GFNA4W7PDP1RD202TQDH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:57.511Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-f0392869101c3eff05ab568a: done: 0.093978 sec","job_status":"done","queue_duration_s":66.466324,"scheduling_latency_s":66.466356,"redis_calls":14,"redis_duration_s":0.004315,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000924,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001992,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001399,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.066999297,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.093978,"completed_at":"2026-09-01T08:50:04.071Z","load_balancing_strategy":"primary","db_duration_s":0.018038,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 280.131176] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:04.074Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"05937f45272a019ae42768dc","created_at":"2026-09-01T08:48:58.087Z","trace_propagation_headers":{"sentry-trace":"f6a6604af6af4e88a20fee572ace0d33-a63abaaba65c4d67","baggage":"sentry-trace_id=f6a6604af6af4e88a20fee572ace0d33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GFNA4W7PDP1RD202TQDH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:58.094Z","job_size_bytes":5,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-05937f45272a019ae42768dc: start","job_status":"start","queue_duration_s":65.979658,"scheduling_latency_s":65.979687} gitlab # [ 280.285820] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:04.228Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"05937f45272a019ae42768dc","created_at":"2026-09-01T08:48:58.087Z","trace_propagation_headers":{"sentry-trace":"f6a6604af6af4e88a20fee572ace0d33-a63abaaba65c4d67","baggage":"sentry-trace_id=f6a6604af6af4e88a20fee572ace0d33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GFNA4W7PDP1RD202TQDH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:58.094Z","job_size_bytes":5,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-05937f45272a019ae42768dc: done: 0.154645 sec","job_status":"done","queue_duration_s":65.979658,"scheduling_latency_s":65.979687,"redis_calls":12,"redis_duration_s":0.0035080000000000003,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000923,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00206,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000525,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.124741,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000524928,"duration_s":0.154645,"completed_at":"2026-09-01T08:50:04.228Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02034,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 280.288133] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:04.230Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"d57312cc7d5fe1f09c3c33f3","created_at":"2026-09-01T08:48:58.120Z","trace_propagation_headers":{"sentry-trace":"f6a6604af6af4e88a20fee572ace0d33-a63abaaba65c4d67","baggage":"sentry-trace_id=f6a6604af6af4e88a20fee572ace0d33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GFNA4W7PDP1RD202TQDH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:58.125Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-d57312cc7d5fe1f09c3c33f3: start","job_status":"start","queue_duration_s":66.104854,"scheduling_latency_s":66.104881} gitlab # [ 280.387685] nixbot[2804]: INFO: ::ffff:127.0.0.1:44648 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.04 seconds) [] gitlab # [ 280.488191] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:04.431Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"d57312cc7d5fe1f09c3c33f3","created_at":"2026-09-01T08:48:58.120Z","trace_propagation_headers":{"sentry-trace":"f6a6604af6af4e88a20fee572ace0d33-a63abaaba65c4d67","baggage":"sentry-trace_id=f6a6604af6af4e88a20fee572ace0d33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GFNA4W7PDP1RD202TQDH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:58.125Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-d57312cc7d5fe1f09c3c33f3: done: 0.200434 sec","job_status":"done","queue_duration_s":66.104854,"scheduling_latency_s":66.104881,"redis_calls":11,"redis_duration_s":0.0032170000000000002,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000917,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001744,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000556,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.164,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.164,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.139712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000423518,"duration_s":0.200434,"completed_at":"2026-09-01T08:50:04.431Z","load_balancing_strategy":"primary","db_duration_s":0.061045,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.546707] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:04.432Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"78be90f64c7a26badd3835d4","created_at":"2026-09-01T08:48:58.128Z","trace_propagation_headers":{"sentry-trace":"f6a6604af6af4e88a20fee572ace0d33-a63abaaba65c4d67","baggage":"sentry-trace_id=f6a6604af6af4e88a20fee572ace0d33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GFNA4W7PDP1RD202TQDH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:58.132Z","job_size_bytes":89,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-78be90f64c7a26badd3835d4: start","job_status":"start","queue_duration_s":66.299808,"scheduling_latency_s":66.299831} gitlab # [ 280.567829] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:04.466Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"78be90f64c7a26badd3835d4","created_at":"2026-09-01T08:48:58.128Z","trace_propagation_headers":{"sentry-trace":"f6a6604af6af4e88a20fee572ace0d33-a63abaaba65c4d67","baggage":"sentry-trace_id=f6a6604af6af4e88a20fee572ace0d33,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GFNA4W7PDP1RD202TQDH","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-01T08:48:58.132Z","job_size_bytes":89,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-78be90f64c7a26badd3835d4: done: 0.03329 sec","job_status":"done","queue_duration_s":66.299808,"scheduling_latency_s":66.299831,"redis_calls":11,"redis_duration_s":0.0025570000000000002,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000663,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001445,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000377981,"duration_s":0.03329,"completed_at":"2026-09-01T08:50:04.465Z","load_balancing_strategy":"primary","db_duration_s":0.005078,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.627117] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:04.467Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 4360d82929f184c6583c9f707539c5b803e775b0 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"efe21710afda345b6c2550ac","created_at":"2026-09-01T08:48:59.769Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-f799e8bc910348d5","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:59.788Z","job_size_bytes":185,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-efe21710afda345b6c2550ac: start","job_status":"start","queue_duration_s":64.679149,"scheduling_latency_s":64.679171} gitlab # [ 280.653900] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:04.484Z","message":"changes: \"0000000000000000000000000000000000000000 4360d82929f184c6583c9f707539c5b803e775b0 refs/heads/master\\n\""} gitlab # [ 280.655317] gitaly[912]: time="2026-09-01T08:50:04.513Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:50:14.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:04.509" grpc.time_ms=4.651 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.656742] gitaly[912]: time="2026-09-01T08:50:04.522Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T08:50:14.018" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:50:04.518" grpc.time_ms=4.188 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.657755] gitaly[912]: time="2026-09-01T08:50:04.559Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-01T08:50:14.053" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:04.553" grpc.time_ms=5.343 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.658924] gitaly[912]: time="2026-09-01T08:50:04.587Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3931 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:50:04.586" grpc.time_ms=0.315 pid=912 span.kind=server system=grpc gitlab # [ 280.731261] gitaly[912]: time="2026-09-01T08:50:04.674Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:50:04.673" grpc.time_ms=0.385 pid=912 span.kind=server system=grpc gitlab # [ 280.748144] gitaly[912]: time="2026-09-01T08:50:04.691Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG diskcache=47bb7caf-e830-4cb4-ac80-4de4924eeeda grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T08:50:14.067" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:50:04.568" pid=912 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 280.749235] gitaly[912]: time="2026-09-01T08:50:04.691Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=52 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4959 command.oublock=8 command.real_time_ms=119 command.spawn_token_fork_ms=0 command.system_time_ms=33 command.user_time_ms=19 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T08:50:14.067" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:50:04.568" grpc.time_ms=123.293 pid=912 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 280.779233] gitaly[912]: time="2026-09-01T08:50:04.722Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-01T08:50:14.221" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:50:04.722" grpc.time_ms=0.14 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.789918] gitaly[912]: time="2026-09-01T08:50:04.733Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=937 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-01T08:50:14.226" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:04.726" grpc.time_ms=6.329 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.850209] gitaly[912]: time="2026-09-01T08:50:04.793Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=339 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-01T08:50:34.089" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:04.789" grpc.time_ms=3.72 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.874382] gitaly[912]: time="2026-09-01T08:50:04.817Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-01T08:50:14.006" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:50:04.817" grpc.time_ms=0.219 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.878956] gitaly[912]: time="2026-09-01T08:50:04.822Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=345 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-01T08:50:14.008" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-01T08:50:04.819" grpc.time_ms=3.101 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.920884] gitaly[912]: time="2026-09-01T08:50:04.864Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:50:14.049" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:04.859" grpc.time_ms=4.463 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.942301] gitaly[912]: time="2026-09-01T08:50:04.885Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:50:14.070" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:04.880" grpc.time_ms=4.57 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 280.971406] gitaly[912]: time="2026-09-01T08:50:04.914Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=955 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-01T08:50:34.205" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:04.906" grpc.time_ms=8.214 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 281.388343] gitaly[912]: time="2026-09-01T08:50:05.331Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-01T08:50:15.010" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:05.330" grpc.time_ms=0.949 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab # [ 281.410176] nixbot[2804]: INFO: ::ffff:127.0.0.1:44652 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab # [ 281.433918] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:05.376Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 4360d82929f184c6583c9f707539c5b803e775b0 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"efe21710afda345b6c2550ac","created_at":"2026-09-01T08:48:59.769Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-f799e8bc910348d5","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:48:59.788Z","job_size_bytes":185,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-efe21710afda345b6c2550ac: done: 0.908756 sec","job_status":"done","queue_duration_s":64.679149,"scheduling_latency_s":64.679171,"gitaly_calls":13,"gitaly_duration_s":0.278601,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.039863,"redis_read_bytes":2103,"redis_write_bytes":21750,"redis_cache_calls":15,"redis_cache_duration_s":0.003711,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.006033,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001306,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7838,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002518,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.016729,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.009566,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.219,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.233,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.552564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000531631,"exclusive_lock_hold_duration_s":0.022612091,"duration_s":0.908756,"completed_at":"2026-09-01T08:50:05.376Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.061231,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 281.489248] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:05.378Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","4360d82929f184c6583c9f707539c5b803e775b0 ee9c51bde01669a7ae604c285f97a1f1057af0cc refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"8bbe46c9ae375e24d954e642","created_at":"2026-09-01T08:49:02.967Z","trace_propagation_headers":{"sentry-trace":"9c5daca9261345c2a57cb397109ad805-acf1fafb4947400a","baggage":"sentry-trace_id=9c5daca9261345c2a57cb397109ad805,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GNCRD4Z8C4PN2PFCPV0V","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:49:02.969Z","job_size_bytes":185,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-8bbe46c9ae375e24d954e642: start","job_status":"start","queue_duration_s":62.409111,"scheduling_latency_s":62.409133} gitlab # [ 281.490312] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:05.389Z","message":"changes: \"4360d82929f184c6583c9f707539c5b803e775b0 ee9c51bde01669a7ae604c285f97a1f1057af0cc refs/heads/master\\n\""} gitlab # [ 281.522065] gitaly[912]: time="2026-09-01T08:50:05.465Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=60 command.spawn_token_fork_ms=56 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:50:15.083" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:05.403" grpc.time_ms=61.254 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 281.544222] gitaly[912]: time="2026-09-01T08:50:05.487Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-01T08:50:15.161" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:50:05.481" grpc.time_ms=5.283 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 281.572366] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:05.514Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"dfb814e96c0944288230438d","created_at":"2026-09-01T08:50:05.510Z","trace_propagation_headers":{"sentry-trace":"9c5daca9261345c2a57cb397109ad805-7f2b37d6f23849a2","baggage":"sentry-trace_id=9c5daca9261345c2a57cb397109ad805,sentry-release=Unknown"},"correlation_id":"01M1E2GNCRD4Z8C4PN2PFCPV0V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"0e0663084944870c31318fe1","job_size_bytes":46,"pid":2816,"job_status":"deduplicated","message":"ProjectCacheWorker JID-dfb814e96c0944288230438d: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 281.609764] gitaly[912]: time="2026-09-01T08:50:05.552Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-01T08:50:15.042" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:05.543" grpc.time_ms=9.681 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 281.636517] gitaly[912]: time="2026-09-01T08:50:05.579Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=652 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-01T08:50:35.074" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:05.574" grpc.time_ms=4.815 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 281.647673] gitaly[912]: time="2026-09-01T08:50:05.590Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=347 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-01T08:50:15.087" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-01T08:50:05.587" grpc.time_ms=2.971 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 281.686080] gitaly[912]: time="2026-09-01T08:50:05.628Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:50:15.123" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:05.623" grpc.time_ms=4.998 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 281.773903] gitaly[912]: time="2026-09-01T08:50:05.717Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-01T08:50:15.215" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:05.715" grpc.time_ms=1.188 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 281.810347] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:05.753Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","4360d82929f184c6583c9f707539c5b803e775b0 ee9c51bde01669a7ae604c285f97a1f1057af0cc refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"8bbe46c9ae375e24d954e642","created_at":"2026-09-01T08:49:02.967Z","trace_propagation_headers":{"sentry-trace":"9c5daca9261345c2a57cb397109ad805-acf1fafb4947400a","baggage":"sentry-trace_id=9c5daca9261345c2a57cb397109ad805,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GNCRD4Z8C4PN2PFCPV0V","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-01T08:49:02.969Z","job_size_bytes":185,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-8bbe46c9ae375e24d954e642: done: 0.374901 sec","job_status":"done","queue_duration_s":62.409111,"scheduling_latency_s":62.409133,"gitaly_calls":7,"gitaly_duration_s":0.19673,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.049095,"redis_read_bytes":500,"redis_write_bytes":11940,"redis_cache_calls":9,"redis_cache_duration_s":0.002335,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001381,"redis_queues_read_bytes":8,"redis_queues_write_bytes":5717,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004521,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.033404,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.007454,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.018,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.018,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.144317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000472406,"exclusive_lock_hold_duration_s":0.070914041,"duration_s":0.374901,"completed_at":"2026-09-01T08:50:05.753Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020041,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 281.818308] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:05.755Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"31dafa98e22cc80a29e0e83a","created_at":"2026-09-01T08:49:04.242Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-8d30d144c7df4df3","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-01T08:49:04.247Z","job_size_bytes":65,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-31dafa98e22cc80a29e0e83a: start","job_status":"start","queue_duration_s":61.507041,"scheduling_latency_s":61.507075} gitlab # [ 281.957292] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:05.899Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"31dafa98e22cc80a29e0e83a","created_at":"2026-09-01T08:49:04.242Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-8d30d144c7df4df3","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-01T08:49:04.247Z","job_size_bytes":65,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-31dafa98e22cc80a29e0e83a: done: 0.144949 sec","job_status":"done","queue_duration_s":61.507041,"scheduling_latency_s":61.507075,"redis_calls":16,"redis_duration_s":0.004678,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.00262,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001543,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000515,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.119419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000349207,"duration_s":0.144949,"completed_at":"2026-09-01T08:50:05.899Z","load_balancing_strategy":"primary","db_duration_s":0.013682,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 282.021263] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:05.964Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"8af40230195398f2b9b47321","created_at":"2026-09-01T08:49:04.248Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-8d30d144c7df4df3","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-01T08:49:04.253Z","job_size_bytes":65,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-8af40230195398f2b9b47321: start","job_status":"start","queue_duration_s":61.710448,"scheduling_latency_s":61.710475} gitlab # [ 282.096244] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:06.038Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"8af40230195398f2b9b47321","created_at":"2026-09-01T08:49:04.248Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-8d30d144c7df4df3","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-01T08:49:04.253Z","job_size_bytes":65,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-8af40230195398f2b9b47321: done: 0.074746 sec","job_status":"done","queue_duration_s":61.710448,"scheduling_latency_s":61.710475,"redis_calls":21,"redis_duration_s":0.0051400000000000005,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002429,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001281,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000342781,"duration_s":0.074746,"completed_at":"2026-09-01T08:50:06.038Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018616,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.098860] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:06.040Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"267dc65c2bb3e6a060ad432a","created_at":"2026-09-01T08:49:04.367Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-8d30d144c7df4df3","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T08:49:04.374Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-267dc65c2bb3e6a060ad432a: start","job_status":"start","queue_duration_s":61.666353,"scheduling_latency_s":61.666375} gitlab # [ 282.226639] gitaly[912]: time="2026-09-01T08:50:06.169Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:50:06.169" grpc.time_ms=0.326 pid=912 span.kind=server system=grpc gitlab # [ 282.260308] gitaly[912]: time="2026-09-01T08:50:06.203Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:50:06.203" grpc.time_ms=0.436 pid=912 span.kind=server system=grpc gitlab # [ 282.294243] gitaly[912]: time="2026-09-01T08:50:06.237Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:50:06.236" grpc.time_ms=0.406 pid=912 span.kind=server system=grpc gitlab # [ 282.312501] gitaly[912]: time="2026-09-01T08:50:06.255Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 diskcache=e85f45c8-f89e-4f48-8c77-edfafbb3e0e0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T08:50:36.047" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:06.147" pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.313696] gitaly[912]: time="2026-09-01T08:50:06.256Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=80 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7725 command.oublock=0 command.real_time_ms=104 command.spawn_token_fork_ms=0 command.system_time_ms=37 command.user_time_ms=43 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T08:50:36.047" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:06.147" grpc.time_ms=108.018 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 282.424110] gitaly[912]: time="2026-09-01T08:50:06.366Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T08:50:36.265" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:06.365" grpc.time_ms=0.698 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.433183] nixbot[2804]: INFO: ::ffff:127.0.0.1:44660 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.03 seconds) [] gitlab # [ 282.452671] gitaly[912]: time="2026-09-01T08:50:06.395Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:36.294" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:06.395" grpc.time_ms=0.675 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.481781] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:06.424Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"267dc65c2bb3e6a060ad432a","created_at":"2026-09-01T08:49:04.367Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-8d30d144c7df4df3","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T08:49:04.374Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2816,"job_status":"rescheduled","message":"PipelineProcessWorker JID-267dc65c2bb3e6a060ad432a: rescheduled"} gitlab # [ 282.491135] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:06.433Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"267dc65c2bb3e6a060ad432a","created_at":"2026-09-01T08:49:04.367Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-8d30d144c7df4df3","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T08:49:04.374Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-267dc65c2bb3e6a060ad432a: done: 0.393311 sec","job_status":"done","queue_duration_s":61.666353,"scheduling_latency_s":61.666375,"gitaly_calls":3,"gitaly_duration_s":0.131074,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.02593,"redis_read_bytes":462,"redis_write_bytes":29294,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001199,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001415,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.009044,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.003983,"redis_queues_read_bytes":39,"redis_queues_write_bytes":15581,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.005796,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000576,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000582,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002136,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.014,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.016,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.203222,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.3404645,"duration_s":0.393311,"completed_at":"2026-09-01T08:50:06.433Z","load_balancing_strategy":"primary","db_duration_s":0.045677,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 282.511855] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:06.435Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"0e0663084944870c31318fe1","created_at":"2026-09-01T08:50:04.524Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-593d2cc599b043d7","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:04.535Z","job_size_bytes":46,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0e0663084944870c31318fe1: start","job_status":"start","queue_duration_s":1.900052,"scheduling_latency_s":1.900085} gitlab # [ 282.577208] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:06.515Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"0e0663084944870c31318fe1","created_at":"2026-09-01T08:50:04.524Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-593d2cc599b043d7","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:04.535Z","job_size_bytes":46,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0e0663084944870c31318fe1: done: 0.079582 sec","job_status":"done","queue_duration_s":1.900052,"scheduling_latency_s":1.900085,"gitaly_calls":2,"gitaly_duration_s":0.01083,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007106,"redis_read_bytes":69,"redis_write_bytes":4882,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001448,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000524,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001683,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002744,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":748,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000707,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000408153,"duration_s":0.079582,"completed_at":"2026-09-01T08:50:06.515Z","load_balancing_strategy":"primary","db_duration_s":0.010023,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 282.657449] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:06.517Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"3391f9a40e18b6a04884eb69","created_at":"2026-09-01T08:50:04.546Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-593d2cc599b043d7","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:04.550Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-3391f9a40e18b6a04884eb69: start","job_status":"start","queue_duration_s":1.967148,"scheduling_latency_s":1.967175} gitlab # [ 282.658957] gitaly[912]: time="2026-09-01T08:50:06.477Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-01T14:50:06.474" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:50:06.475" grpc.time_ms=1.839 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.659721] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:06.588Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"3391f9a40e18b6a04884eb69","created_at":"2026-09-01T08:50:04.546Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-593d2cc599b043d7","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:04.550Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-3391f9a40e18b6a04884eb69: done: 0.071186 sec","job_status":"done","queue_duration_s":1.967148,"scheduling_latency_s":1.967175,"gitaly_calls":1,"gitaly_duration_s":0.017276,"redis_calls":15,"redis_duration_s":0.003695,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000401,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001659,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000598,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.016,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.046905123,"duration_s":0.071186,"completed_at":"2026-09-01T08:50:06.588Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011025,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.739918] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:06.589Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"e0e73eff6b88951af95757d7","created_at":"2026-09-01T08:50:04.704Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-593d2cc599b043d7","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:04.706Z","job_size_bytes":23,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-e0e73eff6b88951af95757d7: start","job_status":"start","queue_duration_s":1.883477,"scheduling_latency_s":1.883499} gitlab # [ 282.741427] gitaly[912]: time="2026-09-01T08:50:06.488Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-01T08:50:36.383" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:06.483" grpc.time_ms=5.264 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.742630] gitaly[912]: time="2026-09-01T08:50:06.555Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG diskcache=c68a9458-5f99-4f4f-8478-b65ae09d5df6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-01T14:50:06.541" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:06.541" pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.808342] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:06.625Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"e0e73eff6b88951af95757d7","created_at":"2026-09-01T08:50:04.704Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-593d2cc599b043d7","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:04.706Z","job_size_bytes":23,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-e0e73eff6b88951af95757d7: done: 0.035465 sec","job_status":"done","queue_duration_s":1.883477,"scheduling_latency_s":1.883499,"gitaly_calls":1,"gitaly_duration_s":0.005578,"redis_calls":16,"redis_duration_s":0.0033220000000000003,"redis_read_bytes":52,"redis_write_bytes":2048,"redis_queues_calls":1,"redis_queues_duration_s":0.000236,"redis_queues_read_bytes":1,"redis_queues_write_bytes":979,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000977,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001466,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000643,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035465,"completed_at":"2026-09-01T08:50:06.625Z","load_balancing_strategy":"primary","db_duration_s":0.002465,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 282.874163] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:06.626Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"e6d6c659dc202555da6f89f3","created_at":"2026-09-01T08:50:05.170Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-593d2cc599b043d7","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2cc2ebfa5ec2a9108510b5bafab221a44b3e6106f00fa2553e2d80d74b326c4","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:05.175Z","job_size_bytes":467,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-e6d6c659dc202555da6f89f3: start","job_status":"start","queue_duration_s":1.451711,"scheduling_latency_s":1.451734} gitlab # [ 282.875296] gitaly[912]: time="2026-09-01T08:50:06.556Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1283 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-01T14:50:06.541" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:06.541" grpc.time_ms=14.464 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.876109] gitaly[912]: time="2026-09-01T08:50:06.617Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-01T08:50:36.513" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:06.613" grpc.time_ms=3.765 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.883563] gitaly[912]: time="2026-09-01T08:50:06.653Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:36.552" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:06.652" grpc.time_ms=0.493 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.942763] gitaly[912]: time="2026-09-01T08:50:06.659Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-01T08:50:16.004" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:06.655" grpc.time_ms=4.134 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.944109] gitaly[912]: time="2026-09-01T08:50:06.768Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T08:50:36.667" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:06.767" grpc.time_ms=0.8 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.944832] gitaly[912]: time="2026-09-01T08:50:06.775Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=325 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-01T08:50:36.671" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:06.771" grpc.time_ms=3.529 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 282.953848] gitaly[912]: time="2026-09-01T08:50:06.854Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:50:16.199" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:06.849" grpc.time_ms=4.471 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 283.461680] nixbot[2804]: INFO: ::ffff:127.0.0.1:45592 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.02 seconds) [] gitlab # [ 283.517801] gitaly[912]: time="2026-09-01T08:50:07.461Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=959 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-01T08:50:37.353" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:07.453" grpc.time_ms=6.965 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.546778] gitaly[912]: time="2026-09-01T08:50:07.489Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GHV0P3MA9E83YDNBH2KG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-01T08:50:37.382" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:07.483" grpc.time_ms=6.819 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.584862] sidekiq-cluster[2816]: {"severity":"WARN","time":"2026-09-01T08:50:07.527Z","class":"CreatePipelineWorker","correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"4360d82929f184c6583c9f707539c5b803e775b0","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"4360d82929f184c6583c9f707539c5b803e775b0","ref":"refs/heads/master"},"retry":0} gitlab # [ 283.591322] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:07.534Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"e6d6c659dc202555da6f89f3","created_at":"2026-09-01T08:50:05.170Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-593d2cc599b043d7","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e2cc2ebfa5ec2a9108510b5bafab221a44b3e6106f00fa2553e2d80d74b326c4","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:05.175Z","job_size_bytes":467,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-e6d6c659dc202555da6f89f3: done: 0.907337 sec","job_status":"done","queue_duration_s":1.451711,"scheduling_latency_s":1.451734,"gitaly_calls":7,"gitaly_duration_s":0.078942,"redis_calls":51,"redis_duration_s":0.010186,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.00081,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004015,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001605,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.00248,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001276,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.793156,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000375746,"duration_s":0.907337,"completed_at":"2026-09-01T08:50:07.534Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033211,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 283.599694] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:07.535Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"4c8c36d3-fa3d-4d27-8646-f6e842e5d1e8\"}"],"class":"WebHookWorker","jid":"4c3696eeab4d8f20285bddc9","created_at":"2026-09-01T08:50:05.304Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-593d2cc599b043d7","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d413ac98731f402944ec6987e2ea61c781c4b0367b8e1c2f613ede32e353ad92","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:05.309Z","job_size_bytes":1751,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-4c3696eeab4d8f20285bddc9: start","job_status":"start","queue_duration_s":2.226326,"scheduling_latency_s":2.226348} gitlab # [ 283.693755] nixbot[2804]: INFO: ::ffff:127.0.0.1:45604 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 283.709552] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:07.652Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"4c8c36d3-fa3d-4d27-8646-f6e842e5d1e8\"}"],"class":"WebHookWorker","jid":"4c3696eeab4d8f20285bddc9","created_at":"2026-09-01T08:50:05.304Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-593d2cc599b043d7","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d413ac98731f402944ec6987e2ea61c781c4b0367b8e1c2f613ede32e353ad92","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:05.309Z","job_size_bytes":1751,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-4c3696eeab4d8f20285bddc9: done: 0.116603 sec","job_status":"done","queue_duration_s":2.226326,"scheduling_latency_s":2.226348,"redis_calls":19,"redis_duration_s":0.003634,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000905,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000268,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001569,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000892,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.036290086999997584,"cpu_s":0.067301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000331047,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.116603,"completed_at":"2026-09-01T08:50:07.652Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002891,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.715208] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:07.654Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"1d6745df6e251876b1007af9","created_at":"2026-09-01T08:50:05.525Z","trace_propagation_headers":{"sentry-trace":"9c5daca9261345c2a57cb397109ad805-7f2b37d6f23849a2","baggage":"sentry-trace_id=9c5daca9261345c2a57cb397109ad805,sentry-release=Unknown"},"correlation_id":"01M1E2GNCRD4Z8C4PN2PFCPV0V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"3391f9a40e18b6a04884eb69","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:05.529Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-1d6745df6e251876b1007af9: start","job_status":"start","queue_duration_s":2.124967,"scheduling_latency_s":2.124989} gitlab # [ 283.779504] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:07.692Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"1d6745df6e251876b1007af9","created_at":"2026-09-01T08:50:05.525Z","trace_propagation_headers":{"sentry-trace":"9c5daca9261345c2a57cb397109ad805-7f2b37d6f23849a2","baggage":"sentry-trace_id=9c5daca9261345c2a57cb397109ad805,sentry-release=Unknown"},"correlation_id":"01M1E2GNCRD4Z8C4PN2PFCPV0V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"3391f9a40e18b6a04884eb69","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:05.529Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-1d6745df6e251876b1007af9: done: 0.037901 sec","job_status":"done","queue_duration_s":2.124967,"scheduling_latency_s":2.124989,"gitaly_calls":1,"gitaly_duration_s":0.009665,"redis_calls":12,"redis_duration_s":0.00303,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001382,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00061,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.020498415,"duration_s":0.037901,"completed_at":"2026-09-01T08:50:07.692Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004438,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.802251] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:07.693Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"7b3ef21f8d15a4233d64cc18","created_at":"2026-09-01T08:50:05.686Z","trace_propagation_headers":{"sentry-trace":"9c5daca9261345c2a57cb397109ad805-7f2b37d6f23849a2","baggage":"sentry-trace_id=9c5daca9261345c2a57cb397109ad805,sentry-release=Unknown"},"correlation_id":"01M1E2GNCRD4Z8C4PN2PFCPV0V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9edcc8b5018b15723bbe0fc5381087894c482d41b42cab0672d1fdecd2423a03","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:05.688Z","job_size_bytes":467,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-7b3ef21f8d15a4233d64cc18: start","job_status":"start","queue_duration_s":2.005719,"scheduling_latency_s":2.00574} gitlab # [ 283.827224] gitaly[912]: time="2026-09-01T08:50:07.679Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V diskcache=fc5f2020-b419-45c4-827b-80b562044499 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-01T14:50:07.673" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:07.673" pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.889562] gitaly[912]: time="2026-09-01T08:50:07.679Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=317 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-01T14:50:07.673" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:07.673" grpc.time_ms=5.416 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.890835] gitaly[912]: time="2026-09-01T08:50:07.744Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:37.642" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:07.742" grpc.time_ms=1.821 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.891992] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:07 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 164 gitlab # [ 283.894843] gitaly[912]: time="2026-09-01T08:50:07.753Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-01T08:50:17.096" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:07.746" grpc.time_ms=6.458 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.895927] gitaly[912]: time="2026-09-01T08:50:07.783Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T08:50:37.682" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:07.782" grpc.time_ms=0.64 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.952979] gitaly[912]: time="2026-09-01T08:50:07.789Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=307 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-01T08:50:37.686" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:07.786" grpc.time_ms=2.651 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.953987] gitlab-workhorse[822]: time="2026-09-01T08:50:07Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 283.955786] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:07 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 68 gitlab # [ 283.956713] gitaly[912]: time="2026-09-01T08:50:07.814Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:50:17.157" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:07.808" grpc.time_ms=5.317 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 283.957665] gitaly[912]: time="2026-09-01T08:50:07.815Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E2JN53ZAAJ6AFB6JG4202P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:07.809" grpc.time_ms=5.894 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 283.961072] gitaly[912]: time="2026-09-01T08:50:07.883Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JNAA4KKZ3WFPWHY8HV5P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:07.875" pid=912 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 283.975383] gitaly[912]: time="2026-09-01T08:50:07.884Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JNAA4KKZ3WFPWHY8HV5P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:07.875" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 283.991283] nixbot[2804]: {"ts": "2026-09-01T08:50:07.906Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 283.994356] gitaly[912]: time="2026-09-01T08:50:07.885Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JNAA4KKZ3WFPWHY8HV5P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:07.875" grpc.time_ms=9.618 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 284.101063] gitaly[912]: time="2026-09-01T08:50:08.044Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JNDCE5N7DG47A1HVNV22 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:38.043" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:08.043" grpc.time_ms=0.609 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.152390] gitaly[912]: time="2026-09-01T08:50:08.095Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2JNDCE5N7DG47A1HVNV22 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:50:18.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:08.090" grpc.time_ms=4.669 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.201528] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4360d82929f184c6583c9f707539c5b803e775b0 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 227 gitlab # [ 284.206561] nixbot[2804]: {"ts": "2026-09-01T08:50:08.149Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4360d82929f184c6583c9f707539c5b803e775b0 \"HTTP/1.1 201 Created\""} gitlab # [ 284.255652] gitaly[912]: time="2026-09-01T08:50:08.198Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JNMRHTP06P216SVA119S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:38.197" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:08.197" grpc.time_ms=0.675 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.324339] gitaly[912]: time="2026-09-01T08:50:08.267Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2JNMRHTP06P216SVA119S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:50:18.261" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:08.261" grpc.time_ms=5.15 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.357650] gitaly[912]: time="2026-09-01T08:50:08.300Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-01T08:50:38.192" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:08.292" grpc.time_ms=7.914 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.386853] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4360d82929f184c6583c9f707539c5b803e775b0 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 177 gitlab # [ 284.387895] nixbot[2804]: {"ts": "2026-09-01T08:50:08.331Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4360d82929f184c6583c9f707539c5b803e775b0 \"HTTP/1.1 201 Created\""} gitlab # [ 284.398931] gitaly[912]: time="2026-09-01T08:50:08.342Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GNCRD4Z8C4PN2PFCPV0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-01T08:50:38.225" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:08.325" grpc.time_ms=16.032 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.408861] sidekiq-cluster[2816]: {"severity":"WARN","time":"2026-09-01T08:50:08.352Z","class":"CreatePipelineWorker","correlation_id":"01M1E2GNCRD4Z8C4PN2PFCPV0V","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"4360d82929f184c6583c9f707539c5b803e775b0","after":"ee9c51bde01669a7ae604c285f97a1f1057af0cc","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"ee9c51bde01669a7ae604c285f97a1f1057af0cc","ref":"refs/heads/master"},"retry":0} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 284.414744] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:08.357Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"7b3ef21f8d15a4233d64cc18","created_at":"2026-09-01T08:50:05.686Z","trace_propagation_headers":{"sentry-trace":"9c5daca9261345c2a57cb397109ad805-7f2b37d6f23849a2","baggage":"sentry-trace_id=9c5daca9261345c2a57cb397109ad805,sentry-release=Unknown"},"correlation_id":"01M1E2GNCRD4Z8C4PN2PFCPV0V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9edcc8b5018b15723bbe0fc5381087894c482d41b42cab0672d1fdecd2423a03","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:05.688Z","job_size_bytes":467,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-7b3ef21f8d15a4233d64cc18: done: 0.663753 sec","job_status":"done","queue_duration_s":2.005719,"scheduling_latency_s":2.00574,"gitaly_calls":7,"gitaly_duration_s":0.079288,"redis_calls":33,"redis_duration_s":0.007486,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000581,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001098,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.004253,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001554,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.524009,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.663753,"completed_at":"2026-09-01T08:50:08.357Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043866,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 284.494446] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:08.360Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"8671beac-e899-4b88-ab60-c66460940b3a\"}"],"class":"WebHookWorker","jid":"162b98d2c82e4f815f118cfe","created_at":"2026-09-01T08:50:05.702Z","trace_propagation_headers":{"sentry-trace":"9c5daca9261345c2a57cb397109ad805-7f2b37d6f23849a2","baggage":"sentry-trace_id=9c5daca9261345c2a57cb397109ad805,sentry-release=Unknown"},"correlation_id":"01M1E2GNCRD4Z8C4PN2PFCPV0V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:935b302222c55dadbb7cf727dbe53369ec2ff862d00698ffdfb75a3ed0b9fb5b","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:05.704Z","job_size_bytes":1749,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-162b98d2c82e4f815f118cfe: start","job_status":"start","queue_duration_s":2.655504,"scheduling_latency_s":2.655535} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.08 seconds) [] gitlab # [ 284.515062] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:08.394Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"8671beac-e899-4b88-ab60-c66460940b3a\"}"],"class":"WebHookWorker","jid":"162b98d2c82e4f815f118cfe","created_at":"2026-09-01T08:50:05.702Z","trace_propagation_headers":{"sentry-trace":"9c5daca9261345c2a57cb397109ad805-7f2b37d6f23849a2","baggage":"sentry-trace_id=9c5daca9261345c2a57cb397109ad805,sentry-release=Unknown"},"correlation_id":"01M1E2GNCRD4Z8C4PN2PFCPV0V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:935b302222c55dadbb7cf727dbe53369ec2ff862d00698ffdfb75a3ed0b9fb5b","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:05.704Z","job_size_bytes":1749,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-162b98d2c82e4f815f118cfe: done: 0.034884 sec","job_status":"done","queue_duration_s":2.655504,"scheduling_latency_s":2.655535,"redis_calls":14,"redis_duration_s":0.003475,"redis_read_bytes":26,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.001001,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001357,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001117,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.005167416000006142,"cpu_s":0.021474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.034884,"completed_at":"2026-09-01T08:50:08.394Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001726,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.570524] nixbot[2804]: INFO: ::ffff:127.0.0.1:45606 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 284.572246] nixbot[2804]: INFO: ::ffff:127.0.0.1:45620 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab # [ 284.574863] gitaly[912]: time="2026-09-01T08:50:08.427Z" level=info msg="finished unary call with code OK" catfile.duration_ms=46 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=46 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JNTG1HTCFFXJCDSQ2551 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:38.379" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:08.380" grpc.time_ms=47.161 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.591313] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:08.396Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"069b5041bf47c6d57b799e49","created_at":"2026-09-01T08:50:06.133Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.137Z","job_size_bytes":23,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-069b5041bf47c6d57b799e49: start","job_status":"start","queue_duration_s":2.25846,"scheduling_latency_s":2.258485} gitlab # [ 284.610588] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:08.482Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"069b5041bf47c6d57b799e49","created_at":"2026-09-01T08:50:06.133Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.137Z","job_size_bytes":23,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-069b5041bf47c6d57b799e49: done: 0.086377 sec","job_status":"done","queue_duration_s":2.25846,"scheduling_latency_s":2.258485,"redis_calls":10,"redis_duration_s":0.0032229999999999997,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000361,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002394,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000468,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054707,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000394464,"duration_s":0.086377,"completed_at":"2026-09-01T08:50:08.482Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011046,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 284.664985] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:08 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 54 gitlab # [ 284.667679] gitlab-workhorse[822]: time="2026-09-01T08:50:08Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 284.670064] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:08 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 66 gitlab # [ 284.672497] gitaly[912]: time="2026-09-01T08:50:08.453Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2JNWFH3G7N7Y1RVTD2WCX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=202 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:08.445" grpc.time_ms=7.775 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 284.688702] gitaly[912]: time="2026-09-01T08:50:08.521Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JNY8GJWBJ4FSE2SDAW0A grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:08.514" pid=912 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 284.701646] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:08.485Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"0e014153256ccc94ecd08b2f","created_at":"2026-09-01T08:50:06.260Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.265Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-0e014153256ccc94ecd08b2f: start","job_status":"start","queue_duration_s":2.219446,"scheduling_latency_s":2.219476} gitlab # [ 284.721688] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:08.520Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"0e014153256ccc94ecd08b2f","created_at":"2026-09-01T08:50:06.260Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.265Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-0e014153256ccc94ecd08b2f: done: 0.035148 sec","job_status":"done","queue_duration_s":2.219446,"scheduling_latency_s":2.219476,"redis_calls":10,"redis_duration_s":0.003759,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000699,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00192,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00114,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000376584,"duration_s":0.035148,"completed_at":"2026-09-01T08:50:08.520Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005378,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.722986] nixbot[2804]: {"ts": "2026-09-01T08:50:08.640Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4360d82929f184c6583c9f707539c5b803e775b0 \"HTTP/1.1 201 Created\""} gitlab # [ 284.724653] gitaly[912]: time="2026-09-01T08:50:08.521Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JNY8GJWBJ4FSE2SDAW0A grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:08.514" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 284.777427] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:08 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 75 gitlab # [ 284.778333] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:08 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4360d82929f184c6583c9f707539c5b803e775b0 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 302 gitlab # [ 284.781313] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:08.521Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"e8252b3cdc32dbc329515842","created_at":"2026-09-01T08:50:06.277Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.282Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-e8252b3cdc32dbc329515842: start","job_status":"start","queue_duration_s":2.239384,"scheduling_latency_s":2.239405} gitlab # [ 284.782073] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:08.568Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"e8252b3cdc32dbc329515842","created_at":"2026-09-01T08:50:06.277Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.282Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-e8252b3cdc32dbc329515842: done: 0.046736 sec","job_status":"done","queue_duration_s":2.239384,"scheduling_latency_s":2.239405,"redis_calls":10,"redis_duration_s":0.005346,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000452,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003816,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024533,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000528838,"duration_s":0.046736,"completed_at":"2026-09-01T08:50:08.568Z","load_balancing_strategy":"primary","db_duration_s":0.007887,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.848206] gitaly[912]: time="2026-09-01T08:50:08.522Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JNY8GJWBJ4FSE2SDAW0A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:08.514" grpc.time_ms=7.438 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 284.848958] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:08.570Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8daef53acc6e84c63f9f256c","created_at":"2026-09-01T08:50:06.294Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.299Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8daef53acc6e84c63f9f256c: start","job_status":"start","queue_duration_s":2.27146,"scheduling_latency_s":2.271489} gitlab # [ 284.865911] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:08.634Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"8daef53acc6e84c63f9f256c","created_at":"2026-09-01T08:50:06.294Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.299Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-8daef53acc6e84c63f9f256c: done: 0.063549 sec","job_status":"done","queue_duration_s":2.27146,"scheduling_latency_s":2.271489,"redis_calls":13,"redis_duration_s":0.010769,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000408,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004849,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.003512,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000384407,"duration_s":0.063549,"completed_at":"2026-09-01T08:50:08.634Z","load_balancing_strategy":"primary","db_duration_s":0.007135,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.937975] gitaly[912]: time="2026-09-01T08:50:08.561Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=53 command.spawn_token_fork_ms=50 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2JNTG1HTCFFXJCDSQ2551 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:50:18.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:08.506" grpc.time_ms=54.473 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 284.946060] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:08.636Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"64f0077cecaae30f2bc65a8a","created_at":"2026-09-01T08:50:06.299Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.304Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-64f0077cecaae30f2bc65a8a: start","job_status":"start","queue_duration_s":2.331934,"scheduling_latency_s":2.331962} gitlab # [ 285.015990] gitaly[912]: time="2026-09-01T08:50:08.618Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=2 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2JP0YXSVZP3QQE8ANBZP1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=194 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:08.610" grpc.time_ms=7.713 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 285.042131] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:08.985Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"64f0077cecaae30f2bc65a8a","created_at":"2026-09-01T08:50:06.299Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.304Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-64f0077cecaae30f2bc65a8a: done: 0.34897 sec","job_status":"done","queue_duration_s":2.331934,"scheduling_latency_s":2.331962,"redis_calls":10,"redis_duration_s":0.319284,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000832,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002881,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.315571,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00062047,"duration_s":0.34897,"completed_at":"2026-09-01T08:50:08.984Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003626,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.099164] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.042Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9507f4eb52a528cb3af21dea","created_at":"2026-09-01T08:50:06.304Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.310Z","job_size_bytes":19,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9507f4eb52a528cb3af21dea: start","job_status":"start","queue_duration_s":2.731952,"scheduling_latency_s":2.731982} gitlab # [ 285.121740] gitaly[912]: time="2026-09-01T08:50:09.053Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JP3CFBYMS1HN5FWHAVET grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:09.047" pid=912 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 285.123164] gitlab-workhorse[822]: time="2026-09-01T08:50:09Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 285.123750] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:09 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 433 gitlab # [ 285.124667] gitaly[912]: time="2026-09-01T08:50:09.053Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JP3CFBYMS1HN5FWHAVET grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:09.047" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 285.125363] gitaly[912]: time="2026-09-01T08:50:09.053Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JP3CFBYMS1HN5FWHAVET grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:09.047" grpc.time_ms=5.89 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 285.208132] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.149Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9507f4eb52a528cb3af21dea","created_at":"2026-09-01T08:50:06.304Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.310Z","job_size_bytes":19,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9507f4eb52a528cb3af21dea: done: 0.107533 sec","job_status":"done","queue_duration_s":2.731952,"scheduling_latency_s":2.731982,"redis_calls":11,"redis_duration_s":0.002691,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000474,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001657,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00056,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.093218,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000395581,"duration_s":0.107533,"completed_at":"2026-09-01T08:50:09.149Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003475,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.213253] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.154Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e864370e133fa748f85bd9c5","created_at":"2026-09-01T08:50:06.312Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.317Z","job_size_bytes":66,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e864370e133fa748f85bd9c5: start","job_status":"start","queue_duration_s":2.836665,"scheduling_latency_s":2.836696} gitlab # [ 285.246063] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.188Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e864370e133fa748f85bd9c5","created_at":"2026-09-01T08:50:06.312Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.317Z","job_size_bytes":66,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e864370e133fa748f85bd9c5: done: 0.03436 sec","job_status":"done","queue_duration_s":2.836665,"scheduling_latency_s":2.836696,"redis_calls":10,"redis_duration_s":0.003049,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000584,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001908,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000557,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000723835,"duration_s":0.03436,"completed_at":"2026-09-01T08:50:09.188Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.339055] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.190Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"118a93d9b195f7179cc40a36","created_at":"2026-09-01T08:50:06.318Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.323Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-118a93d9b195f7179cc40a36: start","job_status":"start","queue_duration_s":2.866798,"scheduling_latency_s":2.866826} gitlab # [ 285.339975] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.226Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"118a93d9b195f7179cc40a36","created_at":"2026-09-01T08:50:06.318Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.323Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-118a93d9b195f7179cc40a36: done: 0.03595 sec","job_status":"done","queue_duration_s":2.866798,"scheduling_latency_s":2.866826,"redis_calls":10,"redis_duration_s":0.0020480000000000003,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00038,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00128,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000388,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000349485,"duration_s":0.03595,"completed_at":"2026-09-01T08:50:09.226Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.394536] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.228Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"1634ca05325a4ae5fd079264","created_at":"2026-09-01T08:50:06.331Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.335Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-1634ca05325a4ae5fd079264: start","job_status":"start","queue_duration_s":2.892374,"scheduling_latency_s":2.892399} gitlab # [ 285.395835] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.263Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"1634ca05325a4ae5fd079264","created_at":"2026-09-01T08:50:06.331Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.335Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-1634ca05325a4ae5fd079264: done: 0.035066 sec","job_status":"done","queue_duration_s":2.892374,"scheduling_latency_s":2.892399,"redis_calls":10,"redis_duration_s":0.002365,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000429,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001464,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000385245,"duration_s":0.035066,"completed_at":"2026-09-01T08:50:09.263Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008135,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.462167] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.264Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"133ac14e5febbc49f459634a","created_at":"2026-09-01T08:50:06.336Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.341Z","job_size_bytes":6,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-133ac14e5febbc49f459634a: start","job_status":"start","queue_duration_s":2.923868,"scheduling_latency_s":2.923892} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.03 seconds) [] gitlab # [ 285.471395] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.290Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"133ac14e5febbc49f459634a","created_at":"2026-09-01T08:50:06.336Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.341Z","job_size_bytes":6,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-133ac14e5febbc49f459634a: done: 0.025651 sec","job_status":"done","queue_duration_s":2.923868,"scheduling_latency_s":2.923892,"redis_calls":10,"redis_duration_s":0.002453,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000511,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001443,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000499,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0146,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000381054,"duration_s":0.025651,"completed_at":"2026-09-01T08:50:09.290Z","load_balancing_strategy":"primary","db_duration_s":0.002579,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.531848] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.292Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"950f285ed193be728e099dc7","created_at":"2026-09-01T08:50:06.341Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.346Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-950f285ed193be728e099dc7: start","job_status":"start","queue_duration_s":2.946379,"scheduling_latency_s":2.94641} gitlab # [ 285.584900] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.323Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"950f285ed193be728e099dc7","created_at":"2026-09-01T08:50:06.341Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.346Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-950f285ed193be728e099dc7: done: 0.030496 sec","job_status":"done","queue_duration_s":2.946379,"scheduling_latency_s":2.94641,"redis_calls":10,"redis_duration_s":0.002342,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000434,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001402,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000506,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000348369,"duration_s":0.030496,"completed_at":"2026-09-01T08:50:09.323Z","load_balancing_strategy":"primary","db_duration_s":0.005366,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.598840] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.324Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"0e97d0c0359580cc4cb44049","created_at":"2026-09-01T08:50:06.351Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.355Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-0e97d0c0359580cc4cb44049: start","job_status":"start","queue_duration_s":2.969034,"scheduling_latency_s":2.969053} gitlab # [ 285.619737] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.382Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"0e97d0c0359580cc4cb44049","created_at":"2026-09-01T08:50:06.351Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.355Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-0e97d0c0359580cc4cb44049: done: 0.057826 sec","job_status":"done","queue_duration_s":2.969034,"scheduling_latency_s":2.969053,"redis_calls":14,"redis_duration_s":0.004966,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001894,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00154,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001532,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000496711,"duration_s":0.057826,"completed_at":"2026-09-01T08:50:09.382Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012075,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.678672] nixbot[2804]: INFO: ::ffff:127.0.0.1:45624 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab # [ 285.681086] gitaly[912]: time="2026-09-01T08:50:09.557Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:50:09.554" grpc.time_ms=2.389 pid=912 span.kind=server system=grpc gitlab # [ 285.737205] gitaly[912]: time="2026-09-01T08:50:09.575Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3890 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:50:09.575" grpc.time_ms=0.361 pid=912 span.kind=server system=grpc gitlab # [ 285.738336] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.384Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"640fd3d4175a47a59d8d23bd","created_at":"2026-09-01T08:50:06.424Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.426Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-640fd3d4175a47a59d8d23bd: start","job_status":"start","queue_duration_s":2.957714,"scheduling_latency_s":2.95774} gitlab # [ 285.739184] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.678Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"640fd3d4175a47a59d8d23bd","created_at":"2026-09-01T08:50:06.424Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.426Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2816,"job_status":"rescheduled","message":"PipelineProcessWorker JID-640fd3d4175a47a59d8d23bd: rescheduled"} gitlab # [ 285.743398] gitaly[912]: time="2026-09-01T08:50:09.595Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:50:09.594" grpc.time_ms=0.306 pid=912 span.kind=server system=grpc gitlab # [ 285.756052] gitaly[912]: time="2026-09-01T08:50:09.600Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 diskcache=d7a490ce-12ac-4ae5-87f2-aca75ce1adc7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T08:50:39.432" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:09.533" pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.817095] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.685Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"640fd3d4175a47a59d8d23bd","created_at":"2026-09-01T08:50:06.424Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-014211ac514d4932","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.426Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-640fd3d4175a47a59d8d23bd: done: 0.30074 sec","job_status":"done","queue_duration_s":2.957714,"scheduling_latency_s":2.95774,"gitaly_calls":3,"gitaly_duration_s":0.074974,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.012866,"redis_read_bytes":149,"redis_write_bytes":18014,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000473,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000777,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.002767,"redis_queues_read_bytes":27,"redis_queues_write_bytes":12313,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.004607,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000202,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00056,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003007,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.085,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.085,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.108748,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.260787894,"duration_s":0.30074,"completed_at":"2026-09-01T08:50:09.684Z","load_balancing_strategy":"primary","db_duration_s":0.109033,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 285.877281] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.686Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"3dc7259f44a5e4f51969e523","created_at":"2026-09-01T08:50:06.497Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-1f4b3378149c4e98","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.502Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-3dc7259f44a5e4f51969e523: start","job_status":"start","queue_duration_s":3.184248,"scheduling_latency_s":3.18427} gitlab # [ 285.879332] gitaly[912]: time="2026-09-01T08:50:09.600Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=72 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7668 command.oublock=0 command.real_time_ms=64 command.spawn_token_fork_ms=0 command.system_time_ms=37 command.user_time_ms=35 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-01T08:50:39.432" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:09.533" grpc.time_ms=67.556 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 285.880863] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.730Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"3dc7259f44a5e4f51969e523","created_at":"2026-09-01T08:50:06.497Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-1f4b3378149c4e98","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:06.502Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-3dc7259f44a5e4f51969e523: done: 0.043309 sec","job_status":"done","queue_duration_s":3.184248,"scheduling_latency_s":3.18427,"redis_calls":19,"redis_duration_s":0.004148,"redis_read_bytes":22,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000762,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000463,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002179,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000744,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00786329,"duration_s":0.043309,"completed_at":"2026-09-01T08:50:09.730Z","load_balancing_strategy":"primary","db_duration_s":0.005609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.950761] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.731Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","4c3696eeab4d8f20285bddc9"],"class":"WebHooks::LogExecutionWorker","jid":"f6b410545f686ac742bf1ab6","created_at":"2026-09-01T08:50:07.639Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-b47336fc335a414b","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9a71522765b75771111a819949d1e55984079b25258c79e14158464424571d4a","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:07.645Z","job_size_bytes":2593,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-f6b410545f686ac742bf1ab6: start","job_status":"start","queue_duration_s":2.08613,"scheduling_latency_s":2.086154} gitlab # [ 286.010377] gitaly[912]: time="2026-09-01T08:50:09.646Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T08:50:39.545" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:09.645" grpc.time_ms=0.758 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 286.011334] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.774Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","4c3696eeab4d8f20285bddc9"],"class":"WebHooks::LogExecutionWorker","jid":"f6b410545f686ac742bf1ab6","created_at":"2026-09-01T08:50:07.639Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-b47336fc335a414b","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9a71522765b75771111a819949d1e55984079b25258c79e14158464424571d4a","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:07.645Z","job_size_bytes":2593,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-f6b410545f686ac742bf1ab6: done: 0.042455 sec","job_status":"done","queue_duration_s":2.08613,"scheduling_latency_s":2.086154,"redis_calls":10,"redis_duration_s":0.002399,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000489,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001385,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000525,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000366807,"duration_s":0.042455,"completed_at":"2026-09-01T08:50:09.774Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006454,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.083241] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.776Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","162b98d2c82e4f815f118cfe"],"class":"WebHooks::LogExecutionWorker","jid":"22710274de1bf133b1ed7e3f","created_at":"2026-09-01T08:50:08.386Z","trace_propagation_headers":{"sentry-trace":"9c5daca9261345c2a57cb397109ad805-e61a3e49531043ff","baggage":"sentry-trace_id=9c5daca9261345c2a57cb397109ad805,sentry-release=Unknown"},"correlation_id":"01M1E2GNCRD4Z8C4PN2PFCPV0V","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:04c50d52602eb947ad20fb2fa3147b56b59feb6ebd8305fbca3b20e14c122643","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:08.388Z","job_size_bytes":2592,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-22710274de1bf133b1ed7e3f: start","job_status":"start","queue_duration_s":1.387564,"scheduling_latency_s":1.38759} gitlab # [ 286.084266] gitaly[912]: time="2026-09-01T08:50:09.670Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2GPVY13MGBAT0Z9K8W897 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:39.569" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:09.670" grpc.time_ms=0.562 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 286.084928] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.800Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","162b98d2c82e4f815f118cfe"],"class":"WebHooks::LogExecutionWorker","jid":"22710274de1bf133b1ed7e3f","created_at":"2026-09-01T08:50:08.386Z","trace_propagation_headers":{"sentry-trace":"9c5daca9261345c2a57cb397109ad805-e61a3e49531043ff","baggage":"sentry-trace_id=9c5daca9261345c2a57cb397109ad805,sentry-release=Unknown"},"correlation_id":"01M1E2GNCRD4Z8C4PN2PFCPV0V","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:04c50d52602eb947ad20fb2fa3147b56b59feb6ebd8305fbca3b20e14c122643","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:08.388Z","job_size_bytes":2592,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-22710274de1bf133b1ed7e3f: done: 0.02461 sec","job_status":"done","queue_duration_s":1.387564,"scheduling_latency_s":1.38759,"redis_calls":7,"redis_duration_s":0.0017009999999999998,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001197,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000504,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02461,"completed_at":"2026-09-01T08:50:09.800Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003809,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.157975] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.802Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"a011f4b68e85f70c4dc260e5","created_at":"2026-09-01T08:50:09.526Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.528Z","job_size_bytes":23,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-a011f4b68e85f70c4dc260e5: start","job_status":"start","queue_duration_s":0.273994,"scheduling_latency_s":0.274017} gitlab # [ 286.159064] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.840Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"a011f4b68e85f70c4dc260e5","created_at":"2026-09-01T08:50:09.526Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.528Z","job_size_bytes":23,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-a011f4b68e85f70c4dc260e5: done: 0.038541 sec","job_status":"done","queue_duration_s":0.273994,"scheduling_latency_s":0.274017,"redis_calls":7,"redis_duration_s":0.0017460000000000002,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001293,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027459,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038541,"completed_at":"2026-09-01T08:50:09.840Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007281,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 286.217743] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.842Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e85ef659a1ab9b606f7ddc60","created_at":"2026-09-01T08:50:09.607Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.609Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e85ef659a1ab9b606f7ddc60: start","job_status":"start","queue_duration_s":0.233148,"scheduling_latency_s":0.233171} gitlab # [ 286.229231] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.867Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e85ef659a1ab9b606f7ddc60","created_at":"2026-09-01T08:50:09.607Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.609Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e85ef659a1ab9b606f7ddc60: done: 0.025373 sec","job_status":"done","queue_duration_s":0.233148,"scheduling_latency_s":0.233171,"redis_calls":7,"redis_duration_s":0.001647,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001238,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000409,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025373,"completed_at":"2026-09-01T08:50:09.867Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004423,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.292929] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.869Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7e01c9e2b8b6cc055b986286","created_at":"2026-09-01T08:50:09.615Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.617Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7e01c9e2b8b6cc055b986286: start","job_status":"start","queue_duration_s":0.25147,"scheduling_latency_s":0.251493} gitlab # [ 286.298152] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.893Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"7e01c9e2b8b6cc055b986286","created_at":"2026-09-01T08:50:09.615Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.617Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-7e01c9e2b8b6cc055b986286: done: 0.024309 sec","job_status":"done","queue_duration_s":0.25147,"scheduling_latency_s":0.251493,"redis_calls":7,"redis_duration_s":0.001467,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001042,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000425,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024309,"completed_at":"2026-09-01T08:50:09.893Z","load_balancing_strategy":"primary","db_duration_s":0.003394,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.348318] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.895Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"fc882a5ceef186b241de17ab","created_at":"2026-09-01T08:50:09.624Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.626Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fc882a5ceef186b241de17ab: start","job_status":"start","queue_duration_s":0.269007,"scheduling_latency_s":0.269031} gitlab # [ 286.414193] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.924Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"fc882a5ceef186b241de17ab","created_at":"2026-09-01T08:50:09.624Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.626Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fc882a5ceef186b241de17ab: done: 0.028317 sec","job_status":"done","queue_duration_s":0.269007,"scheduling_latency_s":0.269031,"redis_calls":10,"redis_duration_s":0.002454,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001168,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000898,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000388,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028317,"completed_at":"2026-09-01T08:50:09.924Z","load_balancing_strategy":"primary","db_duration_s":0.004252,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.416892] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.925Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"101fc4909b23843ec2aeb20a","created_at":"2026-09-01T08:50:09.627Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.628Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-101fc4909b23843ec2aeb20a: start","job_status":"start","queue_duration_s":0.297197,"scheduling_latency_s":0.297221} gitlab # [ 286.437004] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.948Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"101fc4909b23843ec2aeb20a","created_at":"2026-09-01T08:50:09.627Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.628Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-101fc4909b23843ec2aeb20a: done: 0.022807 sec","job_status":"done","queue_duration_s":0.297197,"scheduling_latency_s":0.297221,"redis_calls":7,"redis_duration_s":0.001552,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001167,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000385,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022807,"completed_at":"2026-09-01T08:50:09.948Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003133,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.489097] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.949Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"da94f300f2fa6080c07ad2e8","created_at":"2026-09-01T08:50:09.628Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.630Z","job_size_bytes":19,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-da94f300f2fa6080c07ad2e8: start","job_status":"start","queue_duration_s":0.319034,"scheduling_latency_s":0.319058} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 286.505054] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.980Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"da94f300f2fa6080c07ad2e8","created_at":"2026-09-01T08:50:09.628Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.630Z","job_size_bytes":19,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-da94f300f2fa6080c07ad2e8: done: 0.030459 sec","job_status":"done","queue_duration_s":0.319034,"scheduling_latency_s":0.319058,"redis_calls":8,"redis_duration_s":0.005625,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002505,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00312,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030459,"completed_at":"2026-09-01T08:50:09.980Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00539,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.06 seconds) [] gitlab # [ 286.560432] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:09.981Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ff26fb454e13dff80ab7a957","created_at":"2026-09-01T08:50:09.631Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.632Z","job_size_bytes":66,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ff26fb454e13dff80ab7a957: start","job_status":"start","queue_duration_s":0.348878,"scheduling_latency_s":0.3489} gitlab # [ 286.620647] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:10.004Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ff26fb454e13dff80ab7a957","created_at":"2026-09-01T08:50:09.631Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.632Z","job_size_bytes":66,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ff26fb454e13dff80ab7a957: done: 0.02241 sec","job_status":"done","queue_duration_s":0.348878,"scheduling_latency_s":0.3489,"redis_calls":7,"redis_duration_s":0.0020740000000000003,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001541,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000533,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015248,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02241,"completed_at":"2026-09-01T08:50:10.004Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002636,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.680938] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:10.006Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"b5558d7d371cef4238cb0047","created_at":"2026-09-01T08:50:09.633Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.634Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-b5558d7d371cef4238cb0047: start","job_status":"start","queue_duration_s":0.371544,"scheduling_latency_s":0.371579} gitlab # [ 286.681938] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:10.030Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"b5558d7d371cef4238cb0047","created_at":"2026-09-01T08:50:09.633Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.634Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-b5558d7d371cef4238cb0047: done: 0.024689 sec","job_status":"done","queue_duration_s":0.371544,"scheduling_latency_s":0.371579,"redis_calls":7,"redis_duration_s":0.001921,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00144,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024689,"completed_at":"2026-09-01T08:50:10.030Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004117,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.700698] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:10.032Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"d685f3c2f114412dbba03c36","created_at":"2026-09-01T08:50:09.634Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.636Z","job_size_bytes":6,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-d685f3c2f114412dbba03c36: start","job_status":"start","queue_duration_s":0.396713,"scheduling_latency_s":0.396737} gitlab # [ 286.757266] nixbot[2804]: INFO: ::ffff:127.0.0.1:45634 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab # [ 286.758058] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:10.047Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"d685f3c2f114412dbba03c36","created_at":"2026-09-01T08:50:09.634Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.636Z","job_size_bytes":6,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-d685f3c2f114412dbba03c36: done: 0.014567 sec","job_status":"done","queue_duration_s":0.396713,"scheduling_latency_s":0.396737,"redis_calls":7,"redis_duration_s":0.0014709999999999999,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001068,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000403,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014567,"completed_at":"2026-09-01T08:50:10.047Z","load_balancing_strategy":"primary","db_duration_s":0.00051,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.811524] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:10.049Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"a5d5dcc28215ddbeea1abeeb","created_at":"2026-09-01T08:50:09.636Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.637Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a5d5dcc28215ddbeea1abeeb: start","job_status":"start","queue_duration_s":0.411201,"scheduling_latency_s":0.411224} gitlab # [ 286.812739] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:10.070Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"a5d5dcc28215ddbeea1abeeb","created_at":"2026-09-01T08:50:09.636Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.637Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a5d5dcc28215ddbeea1abeeb: done: 0.021747 sec","job_status":"done","queue_duration_s":0.411201,"scheduling_latency_s":0.411224,"redis_calls":7,"redis_duration_s":0.0016330000000000001,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001213,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00042,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021747,"completed_at":"2026-09-01T08:50:10.070Z","load_balancing_strategy":"primary","db_duration_s":0.002842,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 286.881751] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:10.072Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ab7e45041afa32f5e9ba971b","created_at":"2026-09-01T08:50:09.678Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.680Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ab7e45041afa32f5e9ba971b: start","job_status":"start","queue_duration_s":0.391727,"scheduling_latency_s":0.39175} gitlab # [ 286.883039] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:10.102Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ab7e45041afa32f5e9ba971b","created_at":"2026-09-01T08:50:09.678Z","trace_propagation_headers":{"sentry-trace":"0aa1f1fe2a8d48a699b1839f89ff0f77-5f6740f64ab948b1","baggage":"sentry-trace_id=0aa1f1fe2a8d48a699b1839f89ff0f77,sentry-release=Unknown"},"correlation_id":"01M1E2GPVY13MGBAT0Z9K8W897","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.680Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ab7e45041afa32f5e9ba971b: done: 0.029893 sec","job_status":"done","queue_duration_s":0.391727,"scheduling_latency_s":0.39175,"redis_calls":8,"redis_duration_s":0.002013,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001564,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029893,"completed_at":"2026-09-01T08:50:10.102Z","load_balancing_strategy":"primary","db_duration_s":0.004295,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 286.900821] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:10.106Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"08f633e9a893eb5f772175f2","created_at":"2026-09-01T08:50:09.715Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-5921edf4baac4a4a","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.721Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-08f633e9a893eb5f772175f2: start","job_status":"start","queue_duration_s":0.38476,"scheduling_latency_s":0.384785} gitlab # [ 286.966912] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:10.180Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"08f633e9a893eb5f772175f2","created_at":"2026-09-01T08:50:09.715Z","trace_propagation_headers":{"sentry-trace":"160cb11dd355406a8d198d1f47f8a168-5921edf4baac4a4a","baggage":"sentry-trace_id=160cb11dd355406a8d198d1f47f8a168,sentry-release=Unknown"},"correlation_id":"01M1E2GHV0P3MA9E83YDNBH2KG","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:09.721Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-08f633e9a893eb5f772175f2: done: 0.074292 sec","job_status":"done","queue_duration_s":0.38476,"scheduling_latency_s":0.384785,"redis_calls":11,"redis_duration_s":0.00253,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00041,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001676,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000444,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000410946,"duration_s":0.074292,"completed_at":"2026-09-01T08:50:10.180Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01449,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.029507] gitaly[912]: time="2026-09-01T08:50:10.971Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=1784 command.spawn_token_fork_ms=1781 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2JPKA3RKTJZKEA3MW23Y2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:09.185" grpc.time_ms=1786.254 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 287.030916] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:10 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1841 gitlab # [ 287.196398] gitaly[912]: time="2026-09-01T08:50:11.139Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JRDGH8VSRH1F0SHCWW1N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:11.130" pid=912 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 287.197392] gitaly[912]: time="2026-09-01T08:50:11.140Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1163 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JRDGH8VSRH1F0SHCWW1N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:11.130" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 287.199869] gitaly[912]: time="2026-09-01T08:50:11.140Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1163 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JRDGH8VSRH1F0SHCWW1N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:11.130" grpc.time_ms=10.049 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 287.200742] gitlab-workhorse[822]: time="2026-09-01T08:50:11Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 287.201542] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:11 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 148 gitlab # [ 287.306595] gitaly[912]: time="2026-09-01T08:50:11.249Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JRM88SR5M7BXJX1NXZ68 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:41.248" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:11.249" grpc.time_ms=0.629 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 287.334508] gitaly[912]: time="2026-09-01T08:50:11.277Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E2JRM88SR5M7BXJX1NXZ68 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-01T08:50:41.272" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:11.273" grpc.time_ms=4.549 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 287.343980] gitaly[912]: time="2026-09-01T08:50:11.287Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E2JRM88SR5M7BXJX1NXZ68 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:50:21.202" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:11.282" grpc.time_ms=4.184 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 287.365955] gitaly[912]: time="2026-09-01T08:50:11.309Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JRM88SR5M7BXJX1NXZ68 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-01T08:50:21.228" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:50:11.308" grpc.time_ms=0.213 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 287.374640] gitaly[912]: time="2026-09-01T08:50:11.317Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=621 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JRM88SR5M7BXJX1NXZ68 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-01T08:50:21.233" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:11.313" grpc.time_ms=3.995 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 287.404117] gitaly[912]: time="2026-09-01T08:50:11.347Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2JRM88SR5M7BXJX1NXZ68 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:50:11.346" grpc.time_ms=0.346 pid=912 span.kind=server system=grpc gitlab # [ 287.434114] gitaly[912]: time="2026-09-01T08:50:11.377Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1E2JRM88SR5M7BXJX1NXZ68 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-01T08:50:11.376" grpc.time_ms=0.421 pid=912 span.kind=server system=grpc gitlab # [ 287.451134] gitaly[912]: time="2026-09-01T08:50:11.394Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JRM88SR5M7BXJX1NXZ68 diskcache=716a785b-e4d6-421f-addd-811e1d9326a6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T08:50:21.239" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:50:11.320" pid=912 ref=refs/keep-around/ee9c51bde01669a7ae604c285f97a1f1057af0cc remote_ip="2001:db8:1::1" revision=ee9c51bde01669a7ae604c285f97a1f1057af0cc span.kind=server system=grpc username=root gitlab # [ 287.452091] gitaly[912]: time="2026-09-01T08:50:11.394Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=54 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5301 command.oublock=8 command.real_time_ms=70 command.spawn_token_fork_ms=0 command.system_time_ms=40 command.user_time_ms=14 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JRM88SR5M7BXJX1NXZ68 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-01T08:50:21.239" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-01T08:50:11.320" grpc.time_ms=74.25 pid=912 ref=refs/keep-around/ee9c51bde01669a7ae604c285f97a1f1057af0cc remote_ip="2001:db8:1::1" revision=ee9c51bde01669a7ae604c285f97a1f1057af0cc span.kind=server system=grpc username=root gitlab # [ 287.485108] gitaly[912]: time="2026-09-01T08:50:11.428Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JRM88SR5M7BXJX1NXZ68 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:41.427" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:11.427" grpc.time_ms=0.317 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 287.508899] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:11.451Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"fdc72c258728f53c4feeb45a","created_at":"2026-09-01T08:50:11.445Z","trace_propagation_headers":{"sentry-trace":"43c9cbaccb4b45e1a9e57307e314a8a2-58e80c4327e24eb4","baggage":"sentry-trace_id=43c9cbaccb4b45e1a9e57307e314a8a2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2JRM88SR5M7BXJX1NXZ68","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:11.450Z","job_size_bytes":65,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-fdc72c258728f53c4feeb45a: start","job_status":"start","queue_duration_s":0.001619,"scheduling_latency_s":0.001657} gitlab # [ 287.540266] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:11.483Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"fdc72c258728f53c4feeb45a","created_at":"2026-09-01T08:50:11.445Z","trace_propagation_headers":{"sentry-trace":"43c9cbaccb4b45e1a9e57307e314a8a2-58e80c4327e24eb4","baggage":"sentry-trace_id=43c9cbaccb4b45e1a9e57307e314a8a2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2JRM88SR5M7BXJX1NXZ68","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:11.450Z","job_size_bytes":65,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-fdc72c258728f53c4feeb45a: done: 0.031496 sec","job_status":"done","queue_duration_s":0.001619,"scheduling_latency_s":0.001657,"redis_calls":7,"redis_duration_s":0.001787,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000537,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031496,"completed_at":"2026-09-01T08:50:11.483Z","load_balancing_strategy":"primary","db_duration_s":0.004059,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 287.568190] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:11.484Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"0c5d24d3bc859ddf70213378","created_at":"2026-09-01T08:50:11.451Z","trace_propagation_headers":{"sentry-trace":"43c9cbaccb4b45e1a9e57307e314a8a2-58e80c4327e24eb4","baggage":"sentry-trace_id=43c9cbaccb4b45e1a9e57307e314a8a2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2JRM88SR5M7BXJX1NXZ68","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:11.454Z","job_size_bytes":65,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-0c5d24d3bc859ddf70213378: start","job_status":"start","queue_duration_s":0.030163,"scheduling_latency_s":0.030191} gitlab # [ 287.573178] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:11 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 307 gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc' gitlab # [ 287.593741] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:11.536Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"0c5d24d3bc859ddf70213378","created_at":"2026-09-01T08:50:11.451Z","trace_propagation_headers":{"sentry-trace":"43c9cbaccb4b45e1a9e57307e314a8a2-58e80c4327e24eb4","baggage":"sentry-trace_id=43c9cbaccb4b45e1a9e57307e314a8a2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2JRM88SR5M7BXJX1NXZ68","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:11.454Z","job_size_bytes":65,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-0c5d24d3bc859ddf70213378: done: 0.051622 sec","job_status":"done","queue_duration_s":0.030163,"scheduling_latency_s":0.030191,"redis_calls":10,"redis_duration_s":0.00345,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001237,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.002213,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051622,"completed_at":"2026-09-01T08:50:11.536Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011785,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.664543] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:11.538Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"6cfdc8a21f5d11016515edf0","created_at":"2026-09-01T08:50:11.499Z","trace_propagation_headers":{"sentry-trace":"43c9cbaccb4b45e1a9e57307e314a8a2-58e80c4327e24eb4","baggage":"sentry-trace_id=43c9cbaccb4b45e1a9e57307e314a8a2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2JRM88SR5M7BXJX1NXZ68","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:11.503Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6cfdc8a21f5d11016515edf0: start","job_status":"start","queue_duration_s":0.035196,"scheduling_latency_s":0.035228} gitlab # [ 287.665591] gitaly[912]: time="2026-09-01T08:50:11.605Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2JRM88SR5M7BXJX1NXZ68 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-01T08:50:41.504" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:11.604" grpc.time_ms=0.713 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 287.679974] nixbot[2804]: {"ts": "2026-09-01T08:50:11.623Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ee9c51bde01669a7ae604c285f97a1f1057af0cc \"HTTP/1.1 201 Created\""} gitlab # [ 287.687933] nixbot[2804]: {"ts": "2026-09-01T08:50:11.631Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-ee9c51bde016-4"} gitlab # [ 287.710312] nixbot[2804]: INFO: ::ffff:127.0.0.1:45638 - "GET /api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=ee9c51bde01669a7ae604c285f97a1f1057af0cc', in 0.09 seconds) [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'ee9c51bde01669a7ae604c285f97a1f1057af0cc', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-01T08:50:11.203850Z', 'started_at': None, 'finished_at': None, 'eval_queue': {'position': None, 'blocked_by': [{'owner': 'root', 'project_name': 'test-flake', 'forge': 'gitlab', 'number': 2}]}}] (finished: subtest: push delivers the webhook, in 69.47 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses gitlab # [ 287.721420] gitaly[912]: time="2026-09-01T08:50:11.664Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=1 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JRM88SR5M7BXJX1NXZ68 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:41.561" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:11.662" grpc.time_ms=2.288 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 287.734552] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:11.676Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"6cfdc8a21f5d11016515edf0","created_at":"2026-09-01T08:50:11.499Z","trace_propagation_headers":{"sentry-trace":"43c9cbaccb4b45e1a9e57307e314a8a2-58e80c4327e24eb4","baggage":"sentry-trace_id=43c9cbaccb4b45e1a9e57307e314a8a2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1E2JRM88SR5M7BXJX1NXZ68","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:11.503Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-6cfdc8a21f5d11016515edf0: done: 0.138422 sec","job_status":"done","queue_duration_s":0.035196,"scheduling_latency_s":0.035228,"gitaly_calls":2,"gitaly_duration_s":0.007893,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.007239,"redis_read_bytes":79,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001085,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001297,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000231,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001688,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000386,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.00052,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000947,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.088939,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.108514629,"duration_s":0.138422,"completed_at":"2026-09-01T08:50:11.676Z","load_balancing_strategy":"primary","db_duration_s":0.039659,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 287.745091] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:11.684Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"8e82b27f8305516299cb197b","created_at":"2026-09-01T08:50:11.591Z","trace_propagation_headers":{"sentry-trace":"43c9cbaccb4b45e1a9e57307e314a8a2-dbb8e5c1136041bd","baggage":"sentry-trace_id=43c9cbaccb4b45e1a9e57307e314a8a2,sentry-release=Unknown"},"correlation_id":"01M1E2JRM88SR5M7BXJX1NXZ68","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:11.593Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8e82b27f8305516299cb197b: start","job_status":"start","queue_duration_s":0.090292,"scheduling_latency_s":0.090324} gitlab # [ 287.799306] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:11.717Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"8e82b27f8305516299cb197b","created_at":"2026-09-01T08:50:11.591Z","trace_propagation_headers":{"sentry-trace":"43c9cbaccb4b45e1a9e57307e314a8a2-dbb8e5c1136041bd","baggage":"sentry-trace_id=43c9cbaccb4b45e1a9e57307e314a8a2,sentry-release=Unknown"},"correlation_id":"01M1E2JRM88SR5M7BXJX1NXZ68","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:11.593Z","job_size_bytes":3,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8e82b27f8305516299cb197b: done: 0.034049 sec","job_status":"done","queue_duration_s":0.090292,"scheduling_latency_s":0.090324,"redis_calls":7,"redis_duration_s":0.002013,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001222,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000791,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034049,"completed_at":"2026-09-01T08:50:11.717Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004418,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 287.833098] nixbot[2804]: {"ts": "2026-09-01T08:50:11.730Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-ee9c51bde016-4", "argv0": "bwrap"} gitlab # [ 287.923440] gitaly[912]: time="2026-09-01T08:50:11.866Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JS7EWJFZVMVD5QDMXKX3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:41.864" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:11.865" grpc.time_ms=0.785 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 287.941800] nix-daemon[3027]: accepted connection from pid 3642, user nixbot gitlab # [ 287.953194] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 73 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses, in 0.24 seconds) [{'id': 5, 'sha': 'ee9c51bde01669a7ae604c285f97a1f1057af0cc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T08:50:11.494Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses gitlab # [ 289.024885] gitaly[912]: time="2026-09-01T08:50:12.967Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JT9R0V3R0WQHE4NBAHM2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:42.766" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:12.967" grpc.time_ms=0.626 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 289.046826] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 68 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses, in 0.09 seconds) [{'id': 5, 'sha': 'ee9c51bde01669a7ae604c285f97a1f1057af0cc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T08:50:11.494Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses gitlab # [ 290.116396] gitaly[912]: time="2026-09-01T08:50:14.059Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JVBXVDAQZ7T6ZHWED85F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:44.058" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:14.058" grpc.time_ms=0.498 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 290.145841] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 75 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses, in 0.10 seconds) [{'id': 5, 'sha': 'ee9c51bde01669a7ae604c285f97a1f1057af0cc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T08:50:11.494Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses gitlab # [ 291.207922] gitaly[912]: time="2026-09-01T08:50:15.150Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JWE31H3GWMEZ541GNFH3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:45.149" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:15.150" grpc.time_ms=0.572 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses, in 0.08 seconds) [{'id': 5, 'sha': 'ee9c51bde01669a7ae604c285f97a1f1057af0cc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T08:50:11.494Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] gitlab # [ 291.227061] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 59 gitlab # [ 291.547613] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:15.489Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"96e75d26b216fa5bccdc0ac0","created_at":"2026-09-01T08:50:15.477Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e4bff2542950f06ae35fd1bc931a4fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.487Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-96e75d26b216fa5bccdc0ac0: start","job_status":"start","queue_duration_s":0.002285,"scheduling_latency_s":0.002316} gitlab # [ 291.616750] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:15.559Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"96e75d26b216fa5bccdc0ac0","created_at":"2026-09-01T08:50:15.477Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e4bff2542950f06ae35fd1bc931a4fe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.487Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-96e75d26b216fa5bccdc0ac0: done: 0.069583 sec","job_status":"done","queue_duration_s":0.002285,"scheduling_latency_s":0.002316,"redis_calls":11,"redis_duration_s":0.006263,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000848,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002109,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003306,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016565,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000786133,"exclusive_lock_hold_duration_s":0.006564801,"duration_s":0.069583,"completed_at":"2026-09-01T08:50:15.559Z","load_balancing_strategy":"primary","db_duration_s":0.007854,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.674770] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:15.617Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"fbe669d57899598b1a5b7487","created_at":"2026-09-01T08:50:15.516Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d7e0e14a126420ddbd2a66b0b79a6ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T08:50:15.529Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-fbe669d57899598b1a5b7487: start","job_status":"start","queue_duration_s":0.088598,"scheduling_latency_s":0.088629} gitlab # [ 291.754990] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:15.697Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"fbe669d57899598b1a5b7487","created_at":"2026-09-01T08:50:15.516Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d7e0e14a126420ddbd2a66b0b79a6ca","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-01T08:50:15.529Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-fbe669d57899598b1a5b7487: done: 0.080153 sec","job_status":"done","queue_duration_s":0.088598,"scheduling_latency_s":0.088629,"redis_calls":9,"redis_duration_s":0.023274000000000003,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001809,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.019262,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002203,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.080153,"completed_at":"2026-09-01T08:50:15.697Z","load_balancing_strategy":"primary","db_duration_s":0.007766,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 291.786146] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:15.702Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"ec4e60a2d9f38b7092a76550","created_at":"2026-09-01T08:50:15.538Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0162d570df9cc6fb5044cbbfa93e6eda","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.615Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-ec4e60a2d9f38b7092a76550: start","job_status":"start","queue_duration_s":0.086781,"scheduling_latency_s":0.08681} gitlab # [ 291.863474] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:15.806Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"ec4e60a2d9f38b7092a76550","created_at":"2026-09-01T08:50:15.538Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0162d570df9cc6fb5044cbbfa93e6eda","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.615Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-ec4e60a2d9f38b7092a76550: done: 0.104117 sec","job_status":"done","queue_duration_s":0.086781,"scheduling_latency_s":0.08681,"redis_calls":12,"redis_duration_s":0.012685,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001611,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006044,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00503,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.051,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.008784077,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.104117,"completed_at":"2026-09-01T08:50:15.806Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.051007,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 291.930114] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:15.873Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"fe2b9d3723f072868ad5ae78","created_at":"2026-09-01T08:50:15.645Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a33b8f22299ff057e78da167f4243d70","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.662Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-fe2b9d3723f072868ad5ae78: start","job_status":"start","queue_duration_s":0.210744,"scheduling_latency_s":0.210779} gitlab # [ 292.007194] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:15.950Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"fe2b9d3723f072868ad5ae78","created_at":"2026-09-01T08:50:15.645Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a33b8f22299ff057e78da167f4243d70","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.662Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-fe2b9d3723f072868ad5ae78: done: 0.077114 sec","job_status":"done","queue_duration_s":0.210744,"scheduling_latency_s":0.210779,"redis_calls":9,"redis_duration_s":0.007439,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001555,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002589,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003295,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.077114,"completed_at":"2026-09-01T08:50:15.950Z","load_balancing_strategy":"primary","db_duration_s":0.017519,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 292.064846] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:16.007Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"73deff90125f66b05e9c919e","created_at":"2026-09-01T08:50:15.686Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74a2b1e13cbf78688581ef8616d64e5c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.699Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-73deff90125f66b05e9c919e: start","job_status":"start","queue_duration_s":0.30821,"scheduling_latency_s":0.308238} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses gitlab # [ 292.258132] nix-daemon[3027]: accepted connection from pid 3674, user nixbot gitlab # [ 292.266560] nix-daemon[3027]: accepted connection from pid 3642, user nixbot gitlab # [ 292.294534] gitaly[912]: time="2026-09-01T08:50:16.237Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JXFZKKDJ2TDZGVV306D0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:46.235" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:16.236" grpc.time_ms=0.475 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 292.315265] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 66 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses, in 0.09 seconds) [{'id': 5, 'sha': 'ee9c51bde01669a7ae604c285f97a1f1057af0cc', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-01T08:50:11.494Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] gitlab # [ 292.348373] gitaly[912]: time="2026-09-01T08:50:16.291Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JXHEQN66GQEYP1PEDQRZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:46.290" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:16.290" grpc.time_ms=0.552 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 292.410944] gitaly[912]: time="2026-09-01T08:50:16.354Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2JXHEQN66GQEYP1PEDQRZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:50:26.269" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:16.349" grpc.time_ms=4.675 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 292.461588] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 166 gitlab # [ 292.466392] nixbot[2804]: {"ts": "2026-09-01T08:50:16.409Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ee9c51bde01669a7ae604c285f97a1f1057af0cc \"HTTP/1.1 201 Created\""} gitlab # [ 292.526288] gitaly[912]: time="2026-09-01T08:50:16.469Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JXPW36PSC9GFYMV9VPRR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:46.467" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:16.468" grpc.time_ms=0.574 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 292.582122] gitaly[912]: time="2026-09-01T08:50:16.525Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2JXPW36PSC9GFYMV9VPRR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:50:26.019" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:16.520" grpc.time_ms=5.042 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 292.622121] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 152 gitlab # [ 292.625580] nixbot[2804]: {"ts": "2026-09-01T08:50:16.566Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ee9c51bde01669a7ae604c285f97a1f1057af0cc \"HTTP/1.1 201 Created\""} gitlab # [ 292.680052] gitaly[912]: time="2026-09-01T08:50:16.623Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JXW29RR8DH9HB6HJTH0N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-01T08:50:46.622" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-01T08:50:16.622" grpc.time_ms=0.549 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 292.724497] gitaly[912]: time="2026-09-01T08:50:16.667Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1E2JXW29RR8DH9HB6HJTH0N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-01T08:50:26.162" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-01T08:50:16.663" grpc.time_ms=4.33 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 292.768618] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/ee9c51bde01669a7ae604c285f97a1f1057af0cc HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 133 gitlab # [ 292.769928] nixbot[2804]: {"ts": "2026-09-01T08:50:16.712Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/ee9c51bde01669a7ae604c285f97a1f1057af0cc \"HTTP/1.1 201 Created\""} gitlab # [ 292.909566] gitaly[912]: time="2026-09-01T08:50:16.852Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1E2JY31PB1NH5806Y4ET68P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:16.846" grpc.time_ms=5.875 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 292.910905] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:16 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 52 gitlab # [ 292.945429] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:16.887Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"73deff90125f66b05e9c919e","created_at":"2026-09-01T08:50:15.686Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74a2b1e13cbf78688581ef8616d64e5c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.699Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-73deff90125f66b05e9c919e: done: 0.880086 sec","job_status":"done","queue_duration_s":0.30821,"scheduling_latency_s":0.308238,"redis_calls":564,"redis_duration_s":0.494635,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002711,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003801,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.488123,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.420254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.880086,"completed_at":"2026-09-01T08:50:16.887Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000335,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 293.009517] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:16.889Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"f6cfe25e4869c2387aa3c10d","created_at":"2026-09-01T08:50:15.704Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a21dbe4dec036be9eafaf46e4ee8b6e","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.716Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-f6cfe25e4869c2387aa3c10d: start","job_status":"start","queue_duration_s":1.172745,"scheduling_latency_s":1.172775} gitlab # [ 293.010767] gitlab-workhorse[822]: time="2026-09-01T08:50:16Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 293.011712] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/09/01:08:50:16 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 65 gitlab # [ 293.012779] gitaly[912]: time="2026-09-01T08:50:16.927Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JY4Y316D7K7B0AWNMGMK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:16.917" pid=912 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 293.013841] gitaly[912]: time="2026-09-01T08:50:16.927Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JY4Y316D7K7B0AWNMGMK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:16.917" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 293.014887] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:16.955Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"f6cfe25e4869c2387aa3c10d","created_at":"2026-09-01T08:50:15.704Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a21dbe4dec036be9eafaf46e4ee8b6e","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.716Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-f6cfe25e4869c2387aa3c10d: done: 0.066 sec","job_status":"done","queue_duration_s":1.172745,"scheduling_latency_s":1.172775,"redis_calls":10,"redis_duration_s":0.002493,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000498,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001504,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000491,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000338311,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.003055696},"duration_s":0.066,"completed_at":"2026-09-01T08:50:16.955Z","load_balancing_strategy":"primary","db_duration_s":0.028818,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 293.078083] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:16.957Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"a6b34cc902b3f54c325b2789","created_at":"2026-09-01T08:50:15.728Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31ddbeea94f10c61a6506a0eb6395600","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.779Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-a6b34cc902b3f54c325b2789: start","job_status":"start","queue_duration_s":1.177988,"scheduling_latency_s":1.178008} gitlab # [ 293.140239] gitaly[912]: time="2026-09-01T08:50:16.927Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1E2JY4Y316D7K7B0AWNMGMK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-01T08:50:16.917" grpc.time_ms=9.302 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=912 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 293.141376] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:16.981Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"a6b34cc902b3f54c325b2789","created_at":"2026-09-01T08:50:15.728Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31ddbeea94f10c61a6506a0eb6395600","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.779Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-a6b34cc902b3f54c325b2789: done: 0.023845 sec","job_status":"done","queue_duration_s":1.177988,"scheduling_latency_s":1.178008,"redis_calls":9,"redis_duration_s":0.002323,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000784,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001115,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000424,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023845,"completed_at":"2026-09-01T08:50:16.981Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005155,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 293.209225] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:16.982Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"e5933325754fe3d6bb3c7bb1","created_at":"2026-09-01T08:50:15.793Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"61c6eb53ab0c4d9d702dbb3d2a8e3e59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.804Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-e5933325754fe3d6bb3c7bb1: start","job_status":"start","queue_duration_s":1.177996,"scheduling_latency_s":1.178019} gitlab # [ 293.210057] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:16.993Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 293.211209] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:16.999Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"e5933325754fe3d6bb3c7bb1","created_at":"2026-09-01T08:50:15.793Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"61c6eb53ab0c4d9d702dbb3d2a8e3e59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.804Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-e5933325754fe3d6bb3c7bb1: done: 0.017059 sec","job_status":"done","queue_duration_s":1.177996,"scheduling_latency_s":1.178019,"redis_calls":9,"redis_duration_s":0.002021,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00044,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001102,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017059,"completed_at":"2026-09-01T08:50:16.999Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000476,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses gitlab # [ 293.286911] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:17.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"f57428e4fbfe76c408e9cd34","created_at":"2026-09-01T08:50:15.893Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c5f046b50bf4eed8accc9d60713bf166","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.905Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-f57428e4fbfe76c408e9cd34: start","job_status":"start","queue_duration_s":1.095292,"scheduling_latency_s":1.095315} gitlab # [ 293.288128] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:17.021Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"f57428e4fbfe76c408e9cd34","created_at":"2026-09-01T08:50:15.893Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c5f046b50bf4eed8accc9d60713bf166","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.905Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-f57428e4fbfe76c408e9cd34: done: 0.020479 sec","job_status":"done","queue_duration_s":1.095292,"scheduling_latency_s":1.095315,"redis_calls":11,"redis_duration_s":0.002341,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0004,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001034,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000440838,"exclusive_lock_hold_duration_s":0.003388699,"duration_s":0.020479,"completed_at":"2026-09-01T08:50:17.021Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00202,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 293.363350] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:17.023Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"258f7a476050ead6fe087490","created_at":"2026-09-01T08:50:15.917Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab5a77913f82b106e08816280808ac61","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.926Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-258f7a476050ead6fe087490: start","job_status":"start","queue_duration_s":1.096505,"scheduling_latency_s":1.096525} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/ee9c51bde01669a7ae604c285f97a1f1057af0cc/statuses, in 0.08 seconds) [{'id': 5, 'sha': 'ee9c51bde01669a7ae604c285f97a1f1057af0cc', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-09-01T08:50:11.494Z', 'started_at': None, 'finished_at': '2026-09-01T08:50:16.376Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'ee9c51bde01669a7ae604c285f97a1f1057af0cc', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-09-01T08:50:16.545Z', 'started_at': None, 'finished_at': '2026-09-01T08:50:16.687Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] (finished: subtest: eval and build post commit statuses, in 5.68 seconds) (finished: run the VM test script, in 377.32 seconds) test script finished in 377.34s cleanup kill QemuMachine (pid 46) gitlab # [ 293.364372] sidekiq-cluster[2816]: {"severity":"INFO","time":"2026-09-01T08:50:17.042Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"258f7a476050ead6fe087490","created_at":"2026-09-01T08:50:15.917Z","trace_propagation_headers":{"sentry-trace":"7d9e8594d9134eaeb54bb2b529dda65d-f66188842c7a4304","baggage":"sentry-trace_id=7d9e8594d9134eaeb54bb2b529dda65d,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab5a77913f82b106e08816280808ac61","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-01T08:50:15.926Z","job_size_bytes":2,"pid":2816,"sidekiq_tid":"m8ak","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-258f7a476050ead6fe087490: done: 0.019806 sec","job_status":"done","queue_duration_s":1.096505,"scheduling_latency_s":1.096525,"redis_calls":10,"redis_duration_s":0.002098,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0004,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001264,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000434,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_mqemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14) gitlab # a (finished: cleanup, in 1.60 seconds)