tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab: QEMU running (pid 13) gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab # Formatting '/build/vm-state-gitlab/tmp.IiHBqWDogX', fmt=raw size=1073741824 (finished: start all VMs, in 50.64 seconds) gitlab # mke2fs 1.47.3 (8-Jul-2025) gitlab: waiting for unit setup-gitlab.service gitlab # Discarding device blocks: 0/262144 done gitlab: waiting for the VM to finish booting gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: 0c09c971-8cfb-499e-8932-f91fb94defc9 gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.34 (nixbld@localhost) (gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Jun 1 15:51:08 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/pijifs49ifm1f1rpvf8ycff6j0d88pvq-nixos-system-gitlab-test/init regInfo=/nix/store/6vpz335vdvj22j6jvznz3b7vcbz0y4y0-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000000] kvm-clock: using sched offset of 466899460 cycles gitlab # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000005] tsc: Detected 2400.006 MHz processor gitlab # [ 0.000805] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000831] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000871] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002725] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002736] Using GB pages for direct mapping gitlab # [ 0.002803] RAMDISK: [mem 0xbe4e0000-0xbffcffff] gitlab # [ 0.002812] ACPI: Early table checksum verification disabled gitlab # [ 0.002814] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002818] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002822] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002829] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002831] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.002833] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002834] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002836] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002837] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.002838] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.002839] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.002839] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.002840] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.002840] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003076] No NUMA configuration found gitlab # [ 0.003077] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003079] NODE_DATA(0) allocated [mem 0x1bfffa780-0x1bffffcff] gitlab # [ 0.005836] Zone ranges: gitlab # [ 0.005837] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.005838] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.005839] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005840] Device empty gitlab # [ 0.005841] Movable zone start for each node gitlab # [ 0.005842] Early memory node ranges gitlab # [ 0.005842] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.005843] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.005844] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005845] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.005865] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.006140] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.116591] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.117014] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.117029] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.117059] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.117062] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.117064] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.117065] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.117066] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.117067] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.117070] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.117071] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.117076] TSC deadline timer available gitlab # [ 0.117080] CPU topo: Max. logical packages: 1 gitlab # [ 0.117081] CPU topo: Max. logical dies: 1 gitlab # [ 0.117081] CPU topo: Max. dies per package: 1 gitlab # [ 0.117085] CPU topo: Max. threads per core: 1 gitlab # [ 0.117085] CPU topo: Num. cores per package: 4 gitlab # [ 0.117086] CPU topo: Num. threads per package: 4 gitlab # [ 0.117086] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.117110] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.117127] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.117130] kvm-guest: setup PV sched yield gitlab # [ 0.117145] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.117147] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.117148] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.117149] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.117151] Booting paravirtualized kernel on KVM gitlab # [ 0.117154] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.121613] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.123885] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.123940] kvm-guest: PV spinlocks enabled gitlab # [ 0.123941] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.123944] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/pijifs49ifm1f1rpvf8ycff6j0d88pvq-nixos-system-gitlab-test/init regInfo=/nix/store/6vpz335vdvj22j6jvznz3b7vcbz0y4y0-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.124040] Unknown kernel command line parameters "regInfo=/nix/store/6vpz335vdvj22j6jvznz3b7vcbz0y4y0-closure-info/registration", will be passed to user space. gitlab # [ 0.124053] random: crng init done gitlab # [ 0.124054] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.133124] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.137806] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.137886] software IO TLB: area num 4. gitlab # [ 0.213273] Fallback order for Node 0: 0 gitlab # [ 0.213281] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.213283] Policy zone: Normal gitlab # [ 0.215858] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.225817] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.239858] allocated 12582912 bytes of page_ext gitlab # [ 0.249943] ftrace: allocating 48565 entries in 192 pages gitlab # [ 0.249945] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.250794] Dynamic Preempt: lazy gitlab # [ 0.250996] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.250997] rcu: RCU event tracing is enabled. gitlab # [ 0.250997] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.250999] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.250999] Rude variant of Tasks RCU enabled. gitlab # [ 0.251000] Tracing variant of Tasks RCU enabled. gitlab # [ 0.251000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.251001] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.251032] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.251034] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.251035] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.255371] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.255700] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.255813] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.259392] Console: colour VGA+ 80x25 gitlab # [ 0.259395] printk: legacy console [tty0] enabled gitlab # [ 0.289861] printk: legacy console [ttyS0] enabled gitlab # [ 0.391238] ACPI: Core revision 20250807 gitlab # [ 0.392127] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.393694] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.394696] x2apic enabled gitlab # [ 0.395475] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.396407] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.397630] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.399119] kvm-guest: setup PV IPIs gitlab # [ 0.400691] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.401701] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983d4fda2, max_idle_ns: 440795302975 ns gitlab # [ 0.403404] Calibrating delay loop (skipped) preset value.. 4800.01 BogoMIPS (lpj=2400006) gitlab # [ 0.405491] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.406532] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.407402] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.408405] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.410401] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.412402] Transient Scheduler Attacks: Vulnerable: No microcode gitlab # [ 0.413401] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.414402] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.415401] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.416406] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.417402] active return thunk: srso_alias_return_thunk gitlab # [ 0.418291] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.419401] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.420401] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.421402] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.422401] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.423401] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.424401] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.425402] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.427174] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.428162] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.429161] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.430165] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.431165] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. gitlab # [ 0.461068] Freeing SMP alternatives memory: 44K gitlab # [ 0.461404] pid_max: default: 32768 minimum: 301 gitlab # [ 0.462504] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.463498] landlock: Up and running. gitlab # [ 0.464402] Yama: becoming mindful. gitlab # [ 0.465483] LSM support for eBPF active gitlab # [ 0.466424] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.467546] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.470350] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.470955] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.471406] ... version: 2 gitlab # [ 0.472127] ... bit width: 48 gitlab # [ 0.472404] ... generic counters: 6 gitlab # [ 0.473151] ... generic bitmap: 000000000000003f gitlab # [ 0.473453] ... fixed-purpose counters: 0 gitlab # [ 0.474199] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.475096] ... value mask: 0000ffffffffffff gitlab # [ 0.475404] ... max period: 00007fffffffffff gitlab # [ 0.476329] ... global_ctrl mask: 000000000000003f gitlab # [ 0.477230] signal: max sigframe size: 3376 gitlab # [ 0.477996] rcu: Hierarchical SRCU implementation. gitlab # [ 0.478407] rcu: Max phase no-delay instances is 400. gitlab # [ 0.479484] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.484880] smp: Bringing up secondary CPUs ... gitlab # [ 0.485816] smpboot: x86: Booting SMP configuration: gitlab # [ 0.486413] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.489537] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.491098] smpboot: Total of 4 processors activated (19200.04 BogoMIPS) gitlab # [ 0.491887] Memory: 6013092K/6290904K available (17098K kernel code, 2721K rwdata, 13520K rodata, 3644K init, 3008K bss, 258292K reserved, 0K cma-reserved) gitlab # [ 0.493404] devtmpfs: initialized gitlab # [ 0.494273] x86/mm: Memory block size: 128MB gitlab # [ 0.497471] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.498447] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.499480] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.500505] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.501698] PM: RTC time: 07:43:16, date: 2026-07-21 gitlab # [ 0.505553] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.507736] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.508457] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.510512] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.511415] audit: initializing netlink subsys (disabled) gitlab # [ 0.512426] audit: type=2000 audit(1784619796.866:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.512712] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.514404] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.515404] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.516395] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.517188] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.518207] cpuidle: using governor menu gitlab # [ 0.520663] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.521646] PCI: Using configuration type 1 for base access gitlab # [ 0.522405] PCI: Using configuration type 1 for extended access gitlab # [ 0.523600] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.526746] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.527406] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.528404] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.529405] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.534463] ACPI: Added _OSI(Module Device) gitlab # [ 0.536414] ACPI: Added _OSI(Processor Device) gitlab # [ 0.538410] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.545360] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.552956] ACPI: Interpreter enabled gitlab # [ 0.554474] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.556411] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.558554] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.562411] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.564752] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.577384] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.580460] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.585269] acpiphp: Slot [3] registered gitlab # [ 0.586436] acpiphp: Slot [4] registered gitlab # [ 0.587447] acpiphp: Slot [5] registered gitlab # [ 0.589481] acpiphp: Slot [6] registered gitlab # [ 0.590448] acpiphp: Slot [7] registered gitlab # [ 0.592447] acpiphp: Slot [8] registered gitlab # [ 0.593463] acpiphp: Slot [9] registered gitlab # [ 0.594449] acpiphp: Slot [10] registered gitlab # [ 0.596446] acpiphp: Slot [11] registered gitlab # [ 0.597446] acpiphp: Slot [12] registered gitlab # [ 0.599463] acpiphp: Slot [13] registered gitlab # [ 0.600447] acpiphp: Slot [14] registered gitlab # [ 0.601445] acpiphp: Slot [15] registered gitlab # [ 0.603445] acpiphp: Slot [16] registered gitlab # [ 0.604470] acpiphp: Slot [17] registered gitlab # [ 0.605383] acpiphp: Slot [18] registered gitlab # [ 0.605958] acpiphp: Slot [19] registered gitlab # [ 0.607423] acpiphp: Slot [20] registered gitlab # [ 0.608161] acpiphp: Slot [21] registered gitlab # [ 0.608425] acpiphp: Slot [22] registered gitlab # [ 0.609153] acpiphp: Slot [23] registered gitlab # [ 0.610433] acpiphp: Slot [24] registered gitlab # [ 0.611141] acpiphp: Slot [25] registered gitlab # [ 0.611426] acpiphp: Slot [26] registered gitlab # [ 0.612151] acpiphp: Slot [27] registered gitlab # [ 0.612438] acpiphp: Slot [28] registered gitlab # [ 0.613652] acpiphp: Slot [29] registered gitlab # [ 0.614434] acpiphp: Slot [30] registered gitlab # [ 0.615166] acpiphp: Slot [31] registered gitlab # [ 0.616454] PCI host bridge to bus 0000:00 gitlab # [ 0.617163] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.618406] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.620410] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.623410] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.626410] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.629431] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.633194] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.637708] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.642507] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.648475] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.650463] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.652408] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.654408] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.656408] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.658815] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.664476] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.666494] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.668674] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.669411] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.670648] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.685483] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.686416] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.687425] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.689831] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.691307] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.696436] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 0.697309] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 0.699183] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 0.711381] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 0.714663] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.718244] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 0.719062] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 0.719426] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 0.721661] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.728031] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 0.728410] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 0.729382] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 0.732623] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.735436] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 0.736321] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 0.737220] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 0.740635] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.743371] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 0.745064] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 0.745426] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 0.747674] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.752398] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 0.753324] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 0.754516] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 0.761944] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.773446] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 0.775426] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 0.778465] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 0.781426] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 0.786490] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.794458] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 0.797464] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 0.803765] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.812562] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 0.814421] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 0.816459] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 0.820803] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.829429] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 0.831425] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 0.834477] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 0.847160] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 0.850769] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 0.852794] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 0.855743] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 0.858598] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 0.864727] iommu: Default domain type: Translated gitlab # [ 0.866409] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 0.869959] ACPI: bus type USB registered gitlab # [ 0.871561] usbcore: registered new interface driver usbfs gitlab # [ 0.873436] usbcore: registered new interface driver hub gitlab # [ 0.875437] usbcore: registered new device driver usb gitlab # [ 0.878647] NetLabel: Initializing gitlab # [ 0.880408] NetLabel: domain hash size = 128 gitlab # [ 0.881407] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 0.883571] NetLabel: unlabeled traffic allowed by default gitlab # [ 0.885427] PCI: Using ACPI for IRQ routing gitlab # [ 0.887688] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 0.888400] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 0.888400] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 0.891409] vgaarb: loaded gitlab # [ 0.892269] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 0.893035] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 0.898405] clocksource: Switched to clocksource kvm-clock gitlab # [ 0.900215] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.900958] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.902519] pnp: PnP ACPI init gitlab # [ 0.903749] pnp: PnP ACPI: found 6 devices gitlab # [ 0.914092] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 0.916085] clocksource: Switched to clocksource acpi_pm gitlab # [ 0.917170] NET: Registered PF_INET protocol family gitlab # [ 0.919286] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.937402] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.938957] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.940363] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.944251] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.945608] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.946825] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.948218] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.949641] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.951010] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.952014] NET: Registered PF_XDP protocol family gitlab # [ 0.952857] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 0.953868] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 0.954909] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 0.956052] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.957188] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.958543] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 0.959545] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 0.961986] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 0.964519] PCI: CLS 0 bytes, default 64 gitlab # [ 0.965331] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 0.966416] software IO TLB: mapped [mem 0x00000000ba4e0000-0x00000000be4e0000] (64MB) gitlab # [ 0.966489] Trying to unpack rootfs image as initramfs... gitlab # [ 0.966866] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983d4fda2, max_idle_ns: 440795302975 ns gitlab # [ 0.989652] Initialise system trusted keyrings gitlab # [ 0.990819] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 1.002305] Key type asymmetric registered gitlab # [ 1.003069] Asymmetric key parser 'x509' registered gitlab # [ 1.003985] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 1.005654] io scheduler mq-deadline registered gitlab # [ 1.006468] io scheduler kyber registered gitlab # [ 1.010199] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 1.011523] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 1.013894] Linux agpgart interface v0.103 gitlab # [ 1.014654] ACPI: bus type drm_connector registered gitlab # [ 1.017288] usbcore: registered new interface driver usbserial_generic gitlab # [ 1.018402] usbserial: USB Serial support registered for generic gitlab # [ 1.019418] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 1.019418] Please enable it if your BIOS has the CPPC option. gitlab # [ 1.021708] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 1.023126] drop_monitor: Initializing network drop monitor service gitlab # [ 1.024315] NET: Registered PF_INET6 protocol family gitlab # [ 1.026080] Segment Routing with IPv6 gitlab # [ 1.026763] In-situ OAM (IOAM) with IPv6 gitlab # [ 1.030974] IPI shorthand broadcast: enabled gitlab # [ 1.042496] sched_clock: Marking stable (902036403, 139693571)->(1062585784, -20855810) gitlab # [ 1.047230] registered taskstats version 1 gitlab # [ 1.049847] Loading compiled-in X.509 certificates gitlab # [ 1.072742] Demotion targets for Node 0: null gitlab # [ 1.074736] Key type .fscrypt registered gitlab # [ 1.076150] Key type fscrypt-provisioning registered gitlab # [ 1.078029] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 1.079856] ima: Allocated hash algorithm: sha1 gitlab # [ 1.081481] ima: No architecture policies found gitlab # [ 1.083522] PM: Magic number: 2:11:724 gitlab # [ 1.086565] RAS: Correctable Errors collector initialized. gitlab # [ 1.094765] clk: Disabling unused clocks gitlab # [ 1.095460] PM: genpd: Disabling unused power domains gitlab # [ 1.270866] Freeing initrd memory: 27584K gitlab # [ 1.277995] Freeing unused decrypted memory: 2028K gitlab # [ 1.284242] Freeing unused kernel image (initmem) memory: 3644K gitlab # [ 1.286957] Write protecting the kernel read-only data: 32768k gitlab # [ 1.292032] Freeing unused kernel image (text/rodata gap) memory: 1332K gitlab # [ 1.295997] Freeing unused kernel image (rodata/data gap) memory: 816K gitlab # [ 1.363584] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.364791] Run /init as init process gitlab # [ 1.394837] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.433838] fuse: init (API version 7.45) gitlab # [ 1.442177] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 1.449125] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 1.452548] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 1.490727] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 1.827914] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.840406] systemd[1]: Detected virtualization kvm. gitlab # [ 1.871531] systemd[1]: Detected architecture x86-64. gitlab # [ 1.873733] systemd[1]: Running in initrd. gitlab # [ 1.876310] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.879268] systemd[1]: Hostname set to . gitlab # [ 1.959015] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 2.088538] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 2.089839] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 2.091284] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 2.092464] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 2.094638] systemd[1]: Reached target Path Units. gitlab # [ 2.096837] systemd[1]: Reached target Slice Units. gitlab # [ 2.099067] systemd[1]: Reached target Swaps. gitlab # [ 2.101108] systemd[1]: Reached target Timer Units. gitlab # [ 2.103510] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 2.106515] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 2.109355] systemd[1]: Listening on Journal Sockets. gitlab # [ 2.111904] systemd[1]: Listening on udev Control Socket. gitlab # [ 2.114341] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 2.116305] systemd[1]: Reached target Socket Units. gitlab # [ 2.122418] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 2.128683] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 2.140237] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 2.147863] systemd[1]: Starting Journal Service... gitlab # [ 2.156279] systemd[1]: Starting Load Kernel Modules... gitlab # [ 2.158834] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 2.177016] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 2.191666] netfs: FS-Cache loaded gitlab # [ 2.198337] 9pnet: Installing 9P2000 support gitlab # [ 2.200154] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.202978] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.206148] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 2.207568] systemd-journald[151]: Collecting audit messages is disabled. gitlab # [ 2.208172] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.211664] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.215420] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.218007] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.222405] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.223949] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.227907] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.232099] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 2.247777] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.109495] systemd-modules-load[154]: Using [ 2.249555] systemd[1]: Started Journal Service. gitlab # 4 probe threads gitlab # [ 2.111547] systemd-modules-load[154]: Inserted module 'virtio_gpu' gitlab # [ 2.112631] systemd-modules-load[154]: Inserted module 'dm_mod' gitlab # [ 2.113710] systemd-modules-load[154]: Inserted module 'virtio_balloon' gitlab # [ 2.114845] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 2.116647] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.131326] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 2.132320] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 2.133280] systemd[1]: Reached target Local File Systems. gitlab # [ 2.134075] systemd[1]: Starting Create System Files and Directories... gitlab # [ 2.135492] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 2.147379] systemd[1]: Finished Create System Files and Directories. gitlab # [ 2.158478] systemd-udevd[192]: Using default interface naming scheme 'v260'. gitlab # [ 2.170770] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 2.188518] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.189459] systemd[1]: Reached target System Initialization. gitlab # [ 2.190295] systemd[1]: Reached target Basic System. gitlab # [ 2.415111] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 2.415987] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 2.417173] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 2.418959] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 2.420650] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 2.421961] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 2.423304] usb usb1: Product: UHCI Host Controller gitlab # [ 2.424735] usb usb1: Manufacturer: Linux 6.18.34 uhci_hcd gitlab # [ 2.425641] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 2.435077] hub 1-0:1.0: USB hub found gitlab # [ 2.435894] hub 1-0:1.0: 2 ports detected gitlab # [ 2.438951] SCSI subsystem initialized gitlab # [ 2.443631] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 2.446548] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 2.447330] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 2.451568] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 2.488838] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 2.353058] (udev-worker)[198]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.358187] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.359200] (udev-worker)[204]: eth1: Config file /etc/systemd/networ[ 2.499346] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # k/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.362489] (udev-worker)[204]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.504322] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 2.508534] virtio_blk virtio5: [vdb] 11451080 512-byte logical blocks (5.86 GB/5.46 GiB) gitlab # [ 2.514670] scsi host0: ata_piix gitlab # [ 2.515845] scsi host1: ata_piix gitlab # [ 2.516589] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 2.517805] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.383308] systemd-vconsole-setup[228]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.387822] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.391377] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.393681] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.394581] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.396173] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.422308] systemd-fsck[240]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.426353] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.659875] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 2.675126] ata2: found unknown device (class 0) gitlab # [ 2.678015] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 2.682288] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 2.739059] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 2.766483] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 2.826522] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 2.829554] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 2.832289] usb 1-1: Product: QEMU USB Tablet gitlab # [ 2.833987] usb 1-1: Manufacturer: QEMU gitlab # [ 2.835497] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 2.867866] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 2.880889] usbcore: registered new interface driver usbhid gitlab # [ 2.882989] usbhid: USB HID core driver gitlab # [ 2.893429] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 2.897626] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 3.065093] systemd[1]: Mounting /sysroot... gitlab # [ 3.360007] EXT4-fs (vda): mounted filesystem 0c09c971-8cfb-499e-8932-f91fb94defc9 r/w with ordered data mode. Quota mode: none. gitlab # [ 3.226576] systemd[1]: Mounted /sysroot. gitlab # [ 3.227472] systemd[1]: Reached target Initrd Root File System. gitlab # [ 3.232423] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 3.236584] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 3.239898] systemd[1]: Mounting /sysroot/run... gitlab # [ 3.244459] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.253391] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.396401] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.269817] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.272940] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.273737] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.274491] systemd[1]: Mounted /sysroot/run. gitlab # [ 3.279579] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.280879] systemd-sysroot-fstab-check[269]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.422851] 9p: Installing v9fs 9p2000 file system support gitlab # [ 3.285292] systemd[1]: Reload requested from client PID 269 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.287392] systemd[1]: Reloading... gitlab # [ 3.554174] systemd[1]: Reloading finished in 267 ms. gitlab # [ 3.597908] systemd-sysroot-fstab-check[269]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.716344] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 3.717403] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.718317] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.720868] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.728429] systemd-sysroot-fstab-check[269]: Requesting swap.target/start/replace... gitlab # [ 3.738706] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.741629] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.747115] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.749681] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.771698] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.775652] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.064728] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 4.117573] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 4.119540] systemd[1]: Reached target Initrd File Systems. gitlab # [ 4.124831] systemd[1]: Starting Find NixOS closure... gitlab # [ 4.129794] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 4.163623] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 4.166313] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 4.182662] systemd[1]: Finished Find NixOS closure. gitlab # [ 4.184466] systemd[1]: Reached target Initrd Default Target. gitlab # [ 4.188544] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 4.228465] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 4.230499] systemd[1]: Stopped target Basic System. gitlab # [ 4.232254] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 4.234120] systemd[1]: Stopped target Path Units. gitlab # [ 4.235791] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.238279] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.240989] systemd[1]: Stopped target Slice Units. gitlab # [ 4.242997] systemd[1]: Stopped target Socket Units. gitlab # [ 4.244758] systemd[1]: Stopped target System Initialization. gitlab # [ 4.246711] systemd[1]: Stopped target Swaps. gitlab # [ 4.248270] systemd[1]: Stopped target Timer Units. gitlab # [ 4.249930] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.251897] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.256102] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.258679] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.265261] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 4.271652] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.273331] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.275176] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.279214] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.281369] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.287756] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.289956] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.294867] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.296879] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.301224] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.302818] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.304674] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.306605] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.312058] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.314623] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.317336] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.326114] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.331265] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.338716] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.343539] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.356516] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.361302] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.371249] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.373639] systemd[1]: Closed udev Control Socket. gitlab # [ 4.382290] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.384217] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.386699] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.388837] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.391437] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.399098] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.401544] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.418377] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.422686] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.427760] systemd[1]: Reached target Switch Root. gitlab # [ 4.432393] systemd[1]: Starting NixOS Activation... gitlab # [ 4.560610] initrd-nixos-activation-start[571]: booting system configuration /nix/store/pijifs49ifm1f1rpvf8ycff6j0d88pvq-nixos-system-gitlab-test gitlab # [ 4.604880] initrd-nixos-activation-start[571]: running activation script... gitlab # [ 4.830071] initrd-nixos-activation-start[594]: setting up /etc... gitlab # [ 4.953301] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.957715] systemd[1]: Finished NixOS Activation. gitlab # [ 4.967677] systemd[1]: Starting Switch Root... gitlab # [ 5.000746] systemd[1]: Switching root. gitlab # [ 5.461533] systemd-journald[151]: Received SIGTERM from PID 1 (systemd). gitlab # [ 5.614013] NET: Registered PF_VSOCK protocol family gitlab # [ 5.986815] systemd[1]: systemd 260.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.997023] systemd[1]: Detected virtualization kvm. gitlab # [ 5.998887] systemd[1]: Detected architecture x86-64. gitlab # [ 6.000844] systemd[1]: Detected first boot. gitlab # [ 6.003471] systemd[1]: Initializing machine ID from random generator. gitlab # [ 6.145065] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 6.201990] systemd[1]: Applying preset policy. gitlab # [ 6.335134] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 6.437995] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 6.439405] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 6.441519] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 6.443916] systemd[1]: Created slice Slice /system/getty. gitlab # [ 6.445337] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 6.446913] systemd[1]: Created slice User and Session Slice. gitlab # [ 6.447845] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 6.449094] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 6.450227] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 6.451059] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 6.451878] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 6.452740] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 6.453504] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 6.454359] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 6.455299] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 6.456329] systemd[1]: Reached target Path Units. gitlab # [ 6.457101] systemd[1]: Reached target Remote File Systems. gitlab # [ 6.457981] systemd[1]: Reached target Slice Units. gitlab # [ 6.458754] systemd[1]: Reached target Swaps. gitlab # [ 6.460297] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 6.461802] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 6.464135] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 6.465567] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 6.466612] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 6.467808] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 6.469457] systemd[1]: Listening on udev Control Socket. gitlab # [ 6.473665] systemd[1]: Mounting Huge Pages File System... gitlab # [ 6.475717] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 6.477845] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 6.479759] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 6.487420] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 6.488539] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 6.491606] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 6.492615] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 6.494247] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.495967] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.498669] systemd[1]: Mounting FUSE Control File System... gitlab # [ 6.499558] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.503886] systemd[1]: Starting Journal Service... gitlab # [ 6.507814] systemd[1]: Starting Load Kernel Modules... gitlab # [ 6.517050] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 6.526896] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 6.532488] systemd-journald[788]: Collecting audit messages is enabled. gitlab # [ 6.533848] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 6.541580] loop: module loaded gitlab # [ 6.545936] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 6.418618] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 6.559159] systemd[1]: Started Journal Service. gitlab # [ 6.420073] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 6.421567] systemd[1]: systemd-journald.service: Deactivated successfully.[ 6.561583] EXT4-fs (vda): re-mounted 0c09c971-8cfb-499e-8932-f91fb94defc9. gitlab # gitlab # [ 6.423822] systemd-modules-load[789]: Using 3 probe threads gitlab # [ 6.425046] systemd-modules-load[789]: Inserted module 'loop' gitlab # [ 6.426228] systemd-modules-load[789]: Inserted module 'tls' gitlab # [ 6.427300] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 6.428402] systemd[1]: Mounted Huge Pages File System. gitlab # [ 6.429377] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 6.430841] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 6.432066] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 6.433873] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 6.435220] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 6.436286] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 6.437111] systemd[1]: Mounted FUSE Control File System. gitlab # [ 6.437958] systemd[1]: Finished Load Kernel Modules. gitlab # [ 6.438847] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 6.439850] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 6.440829] systemd[1]: Starting Firewall... gitlab # [ 6.444068] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 6.445062] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.447276] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 6.453033] systemd-oomd[790]: No swap; memory pressure usage will be degraded gitlab # [ 6.455527] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 6.456528] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 6.457682] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 6.459081] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 6.460520] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 6.601938] systemd-journald[788]: Received client request to flush runtime journal. gitlab # [ 6.506229] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 6.507450] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 6.508512] systemd[1]: Reached target First Boot Complete. gitlab # [ 6.509487] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 6.510860] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 6.511833] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 6.513054] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 6.514042] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 6.515310] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 6.549969] systemd-udevd[838]: Using default interface naming scheme 'v260'. gitlab # [ 6.578823] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 6.582265] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.658152] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.679714] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 6.683966] (udev-worker)[912]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 6.686202] (udev-worker)[912]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.694121] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 6.711057] systemd[1]: Finished Firewall. gitlab # [ 6.713908] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 6.715174] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.716725] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.719041] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.720713] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 6.722043] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 6.745973] (udev-worker)[907]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.889763] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 6.892594] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 6.904900] Console: switching to colour dummy device 80x25 gitlab # [ 6.905629] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 6.905941] ACPI: button: Power Button [PWRF] gitlab # [ 6.906120] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 6.915272] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 6.916367] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 6.916437] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 6.917943] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 6.925660] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4 gitlab # [ 6.928491] fbcon: bochs-drmdrmfb (fb0) is primary device gitlab # [ 6.943403] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 6.944014] Console: switching to colour frame buffer device 160x50 gitlab # [ 6.945515] rtc_cmos 00:05: registered as rtc0 gitlab # [ 6.946347] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 6.948311] rtc_cmos 00:05: setting system clock to 2026-07-21T07:43:23 UTC (1784619803) gitlab # [ 6.949791] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 6.950580] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 6.950989] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 6.961115] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 6.974084] FDC 0 is a S82078B gitlab # [ 6.980091] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 gitlab # [ 6.984036] mousedev: PS/2 mouse device common for all mice gitlab # [ 6.987048] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 6.893773] systemd[1]: Starting Virtual Console Setup... gitlab # [ 7.048756] ppdev: user-space parallel port driver gitlab # [ 7.075037] erofs (device vdb): failed to read inode meta block (nid: 108841984): -4 gitlab # [ 6.938276] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.939344] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.945499] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.995239] systemd-vconsole-setup[962]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 6.997201] systemd[1]: Finished Virtual Console Setup. gitlab # [ 7.168282] kvm_amd: TSC scaling supported gitlab # [ 7.168852] kvm_amd: Nested Virtualization enabled gitlab # [ 7.169329] kvm_amd: Nested Paging enabled gitlab # [ 7.169847] kvm_amd: LBR virtualization supported gitlab # [ 7.170379] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 7.170986] kvm_amd: Virtual GIF supported gitlab # [ 7.171402] kvm_amd: Virtual NMI enabled gitlab # [ 7.204312] EDAC MC: Ver: 3.0.0 gitlab # [ 7.303234] systemd[1]: Mounting /run/wrappers... gitlab # [ 7.328098] systemd[1]: Mounted /run/wrappers. gitlab # [ 7.330070] systemd[1]: Reached target Local File Systems. gitlab # [ 7.331913] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 7.334203] systemd[1]: Starting register-nix-paths.service... gitlab # [ 7.336524] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 7.338506] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 7.343316] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 7.345581] systemd[1]: Starting Create System Files and Directories... gitlab # [ 7.387246] systemd[1]: Finished Create System Files and Directories. gitlab # [ 7.393451] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 7.397409] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 7.403373] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 7.406936] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 7.426715] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 7.439317] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 7.440795] systemd[1]: Starting Update is Completed... gitlab # [ 7.456221] systemd[1]: Finished Update is Completed. gitlab # [ 7.495672] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 7.496861] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.497740] systemd[1]: suid-sgid-wrappers.service: Consumed 128ms CPU time over 159ms wall clock time, 4.5M memory peak, 1.1M read from disk. gitlab # [ 7.738372] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.740379] systemd[1]: Reached target System Initialization. gitlab # [ 7.742187] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.744375] systemd[1]: Started logrotate.timer. gitlab # [ 7.745832] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.747806] systemd[1]: Reached target Timer Units. gitlab # [ 7.749350] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.751218] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.754521] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.756648] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 7.758660] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.761172] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.763191] systemd[1]: Reached target Socket Units. gitlab # [ 7.831847] dbus-broker-launch[1054]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.834819] dbus-broker-launch[1054]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.837723] dbus-broker-launch[1054]: Invalid user-name in /nix/store/89zj9nmrajvcy9c3c08wv0ir9ww7lgmq-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.843641] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.858078] dbus-broker-launch[1054]: Ready gitlab # [ 7.861849] systemd[1]: Reached target Basic System. gitlab # [ 7.864730] systemd[1]: Started backdoor.service. gitlab # [ 7.867641] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.870339] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.876441] systemd[1]: Starting Logrotate configuration check... gitlab # [ 7.887131] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.893317] systemd[1]: Starting Post-Boot Actions... gitlab # [ 7.896827] logrotate[1059]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # [ 7.901891] logrotate[1059]: reading config file /nix/store/sn4bd2nqi3yg4r51djm9i2rmfgbwvmhb-logrotate.conf gitlab # [ 7.905386] logrotate[1059]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 7.908057] logrotate[1059]: Reading state from file: /var/lib/logrotate.status gitlab # connecting to host... gitlab # [ 7.911346] logrotate[1059]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.913602] logrotate[1059]: Allocating hash table for state file, size 64 entries gitlab # [ 7.915649] logrotate[1059]: Handling 4 logs gitlab # [ 7.916895] logrotate[1059]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.920964] logrotate[1059]: considering log /var/log/btmp gitlab # [ 7.922623] logrotate[1059]: Creating new state gitlab # [ 7.923994] logrotate[1059]: Now: 2026-07-21 07:43 gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab: connected to guest root shell gitlab # [ 7.925453] logrotate[1059]: Last rotated at 2026-07-21 07:00 gitlab: (connecting took 8.54 seconds) gitlab # [ 7.927157] logrotate[1059]: log does not need rotating (log has already been rotated) gitlab: (finished: waiting for the VM to finish booting, in 8.54 seconds) gitlab # [ 7.929187] logrotate[1059]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.932131] logrotate[1059]: considering log /var/log/wtmp gitlab # [ 7.932822] logrotate[1059]: Creating new state gitlab # [ 7.933457] logrotate[1059]: Now: 2026-07-21 07:43 gitlab # [ 7.934137] logrotate[1059]: Last rotated at 2026-07-21 07:00 gitlab # [ 7.934894] logrotate[1059]: log does not need rotating (log has already been rotated) gitlab # [ 7.935893] logrotate[1059]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 7.937568] logrotate[1059]: switching euid from 0 to 165 and egid from 0 to 165 (pid 1059) gitlab # [ 7.938786] logrotate[1059]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.939739] logrotate[1059]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.940837] logrotate[1059]: Creating new state gitlab # [ 7.941488] logrotate[1059]: switching euid from 165 to 0 and egid from 165 to 0 (pid 1059) gitlab # [ 7.942570] logrotate[1059]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.944252] logrotate[1059]: switching euid from 0 to 60 and egid from 0 to 60 (pid 1059) gitlab # [ 7.945345] logrotate[1059]: considering log /var/log/nginx/*.log gitlab # [ 7.946274] logrotate[1059]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.947321] logrotate[1059]: Creating new state gitlab # [ 7.947934] logrotate[1059]: switching euid from 60 to 0 and egid from 60 to 0 (pid 1059) gitlab # [ 7.949081] systemd[1]: Started Reset console on configuration changes. gitlab # [ 7.950376] nsncd[1060]: Jul 21 07:43:24.631 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.952495] systemd[1]: Starting resolvconf update... gitlab # [ 7.953407] systemd[1]: Finished Logrotate configuration check. gitlab # [ 7.954433] systemd[1]: Finished Post-Boot Actions. gitlab # [ 7.955278] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.956333] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 7.957552] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.958490] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.959423] systemd[1]: Starting User Login Management... gitlab # [ 7.986350] systemd-logind[1091]: New seat seat0. gitlab # [ 7.987981] systemd-logind[1091]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 7.988576] systemd-logind[1091]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 7.989764] systemd-logind[1091]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 7.991337] systemd[1]: Started User Login Management. gitlab # [ 7.993504] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.993656] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.994488] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.995054] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.996897] systemd[1]: Starting linger-users.service... gitlab # [ 7.997812] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 8.000287] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 8.003670] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 8.003888] systemd[1]: nscd.service: Consumed 26ms CPU time over 113ms wall clock time, 7.9M memory peak, 5.7M read from disk. gitlab # [ 8.010565] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 8.013394] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 8.014253] systemd[1]: Finished linger-users.service. gitlab # [ 8.018659] systemd[1]: Finished resolvconf update. gitlab # [ 8.019560] systemd[1]: Reached target Preparation for Network. gitlab # [ 8.022039] systemd[1]: Starting DHCP Client... gitlab # [ 8.023957] systemd[1]: Starting Address configuration of eth1... gitlab # [ 8.027322] systemd[1]: Starting Extra networking commands.... gitlab # [ 8.029243] nsncd[1154]: Jul 21 07:43:24.720 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 8.029562] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 8.034140] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 8.035345] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 8.050107] network-addresses-eth1-start[1162]: adding address 192.168.1.1/24... done gitlab # [ 8.053725] network-addresses-eth1-start[1162]: adding address 2001:db8:1::1/64... done gitlab # [ 8.057749] systemd[1]: Finished Address configuration of eth1. gitlab # [ 8.085217] systemd[1]: Finished Extra networking commands.. gitlab # [ 8.086327] systemd[1]: Reached target Network. gitlab # [ 8.087196] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 8.089497] systemd[1]: Starting Nginx Web Server... gitlab # [ 8.094116] systemd[1]: Starting PostgreSQL Server... gitlab # [ 8.096534] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 8.098673] dhcpcd[1189]: dhcpcd-10.3.2 starting gitlab # [ 8.100314] systemd[1]: Starting Permit User Sessions... gitlab # [ 8.101101] dhcpcd[1235]: dev: loaded udev gitlab # [ 8.246620] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 8.247249] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 8.119512] systemd[1]: Finished Permit User Sessions. gitlab # [ 8.123351] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 8.127822] systemd[1]: Started Getty on tty1. gitlab # [ 8.129640] systemd[1]: Reached target Login Prompts. gitlab # [ 8.335244] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 8.342326] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 8.343834] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 8.345961] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 8.347295] cfg80211: failed to load regulatory.db gitlab # [ 8.227210] nginx-pre-start[1255]: nginx: the configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf syntax is ok gitlab # [ 8.227761] nginx-pre-start[1255]: nginx: configuration file /nix/store/nlpb2pnbsk0921apkv5yml2qxbm30d7w-nginx.conf test is successful gitlab # [ 8.230515] postgresql-pre-start[1257]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 8.230930] postgresql-pre-start[1257]: This user must also own the server process.[ 8.372123] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # gitlab # [ 8.234194] redis-server[1253]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 8.234546] redis-server[1253]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 8.234924] redis-server[1253]: Redis version=8.8.0, bits=64, commit=00000000, modified=0, pid=1253, just started gitlab # [ 8.235746] redis-server[1253]: Configuration loaded gitlab # [ 8.238031] dhcpcd[1235]: eth0: waiting for carrier gitlab # [ 8.238720] postgresql-pre-start[1257]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 8.239109] postgresql-pre-start[1257]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 8.239577] postgresql-pre-start[1257]: The default text search configuration will be set to "english". gitlab # [ 8.240051] postgresql-pre-start[1257]: Data page checksums are disabled. gitlab # [ 8.240619] postgresql-pre-start[1257]: fixing permissions on existing directory /var/lib/postgresql/17 ... ok gitlab # [ 8.241171] postgresql-pre-start[1257]: creating subdirectories ... ok gitlab # [ 8.241687] postgresql-pre-start[1257]: selecting dynamic shared memory implementation ... posix gitlab # [ 8.242330] dhcpcd[1235]: eth0: carrier acquired gitlab # [ 8.242810] dhcpcd[1235]: DUID 00:01:00:01:31:f1:df:9c:52:54:00:12:34:56 gitlab # [ 8.243584] dhcpcd[1235]: eth0: IAID 00:12:34:56 gitlab # [ 8.244094] dhcpcd[1235]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 8.245733] systemd[1]: Started Nginx Web Server. gitlab # [ 8.252244] redis[1253]: monotonic clock: POSIX clock_gettime gitlab # [ 8.255207] redis[1253]: Running mode=standalone, port=0. gitlab # [ 8.257251] redis[1253]: Server initialized gitlab # [ 8.258247] redis[1253]: Ready to accept connections unix gitlab # [ 8.259186] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 8.260177] redis[1253]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 8.275690] postgresql-pre-start[1257]: selecting default "max_connections" ... 100 gitlab # [ 8.306560] postgresql-pre-start[1257]: selecting default "shared_buffers" ... 128MB gitlab # [ 8.520290] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.631150] postgresql-pre-start[1257]: selecting default time zone ... UTC gitlab # [ 8.632627] postgresql-pre-start[1257]: creating configuration files ... ok gitlab # [ 8.752401] gitlab-workhorse[1289]: time="2026-07-21T07:43:25Z" level=info msg=Starting build_time=19700101.000000 version=18.11.4 gitlab # [ 8.752600] gitlab-workhorse[1289]: time="2026-07-21T07:43:25Z" level=info msg="Using redis/go-redis" gitlab # [ 8.753104] gitlab-workhorse[1289]: time="2026-07-21T07:43:25Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 8.787828] postgresql-pre-start[1257]: running bootstrap script ... ok gitlab # [ 9.124170] dhcpcd[1235]: eth0: soliciting a DHCP lease gitlab # [ 9.273280] NET: Registered PF_PACKET protocol family gitlab # [ 9.142235] dhcpcd[1235]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 9.148341] dhcpcd[1235]: eth0: probing address 10.0.2.15/24 gitlab # [ 9.221540] postgresql-pre-start[1257]: performing post-bootstrap initialization ... ok gitlab # [ 9.441201] postgresql-pre-start[1257]: syncing data to disk ... ok gitlab # [ 9.443438] postgresql-pre-start[1257]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 9.446540] postgresql-pre-start[1257]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 9.451431] postgresql-pre-start[1257]: Success. You can now start the database server using: gitlab # [ 9.454147] postgresql-pre-start[1257]: pg_ctl -D /var/lib/postgresql/17 -l logfile start gitlab # [ 9.556068] postgres[1316]: [1316] LOG: starting PostgreSQL 17.10 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 9.559834] postgres[1316]: [1316] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 9.562727] postgres[1316]: [1316] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 9.565622] postgres[1316]: [1316] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 9.573801] postgres[1322]: [1322] LOG: database system was shut down at 2026-07-21 07:43:25 GMT gitlab # [ 9.584564] postgres[1316]: [1316] LOG: database system is ready to accept connections gitlab # [ 9.589287] systemd[1]: Started PostgreSQL Server. gitlab # [ 9.594219] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 9.760216] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1331]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 9.760950] 119c5p75zqwmpdrsjsd5cszxwrq2hgfq-gitlab-config[1331]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 9.771232] postgresql-setup-start[1338]: CREATE DATABASE gitlab # [ 9.794912] postgresql-setup-start[1353]: CREATE ROLE gitlab # [ 9.800577] systemd[1]: Finished gitlab-config.service. gitlab # [ 9.803233] systemd[1]: Started gitaly.service. gitlab # [ 9.807144] postgresql-setup-start[1356]: ALTER DATABASE gitlab # [ 9.823970] postgresql-setup-start[1364]: CREATE ROLE gitlab # [ 9.825838] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 9.827119] systemd[1]: Reached target PostgreSQL. gitlab # [ 9.828508] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 9.829835] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 9.914760] gitlab-postgresql-start[1374]: CREATE DATABASE gitlab # [ 9.944627] gitlab-postgresql-start[1379]: CREATE EXTENSION gitlab # [ 9.983306] gitlab-postgresql-start[1381]: CREATE EXTENSION gitlab # [ 9.985940] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 10.048560] dhcpcd[1235]: eth0: soliciting an IPv6 router gitlab # [ 10.049560] dhcpcd[1235]: eth0: Router Advertisement from fe80::2 gitlab # [ 10.050494] dhcpcd[1235]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 10.051492] dhcpcd[1235]: eth0: adding route to fec0::/64 gitlab # [ 10.052380] dhcpcd[1235]: eth0: adding default route via fe80::2 gitlab # [ 10.310610] gitaly[1358]: time="2026-07-21T07:43:27.001Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/2wcq0nlg508fs5wq7jnpg1lw2pib950g-gitaly.toml' and fix the errors that are printed." pid=1358 gitlab # [ 10.314139] gitaly[1358]: time="2026-07-21T07:43:27.001Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=1358 gitlab # [ 10.315672] gitaly[1358]: time="2026-07-21T07:43:27.001Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=1358 gitlab # [ 10.317589] gitaly[1358]: time="2026-07-21T07:43:27.001Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=1358 gitlab # [ 10.319241] gitaly[1358]: time="2026-07-21T07:43:27.001Z" level=info msg="Starting Gitaly" pid=1358 version=18.11.4 gitlab # [ 10.468116] gitaly[1358]: time="2026-07-21T07:43:27.159Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=157 pid=1358 gitlab # [ 10.468409] gitaly[1358]: time="2026-07-21T07:43:27.159Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=1358 gitlab # [ 10.468504] gitaly[1358]: time="2026-07-21T07:43:27.159Z" level=info msg="finished initializing command factory" duration_ms=0 pid=1358 gitlab # [ 10.469116] gitaly[1358]: time="2026-07-21T07:43:27.159Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-1358/git-exec-2793342812.d/git pid=1358 gitlab # [ 10.472137] gitaly[1358]: time="2026-07-21T07:43:27.163Z" level=info msg="finished detecting git version" duration_ms=3 pid=1358 gitlab # [ 10.472867] gitaly[1358]: time="2026-07-21T07:43:27.163Z" level=info msg="using Git version" pid=1358 version=2.53.g gitlab # [ 10.473624] gitaly[1358]: time="2026-07-21T07:43:27.163Z" level=info msg="clearing disk cache object folder" pid=1358 storage=default gitlab # [ 10.474197] gitaly[1358]: time="2026-07-21T07:43:27.163Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3191043274 pid=1358 storage=default gitlab # [ 10.474772] gitaly[1358]: time="2026-07-21T07:43:27.164Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3191043274 pid=1358 storage=default gitlab # [ 10.475810] gitaly[1358]: time="2026-07-21T07:43:27.164Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=1358 gitlab # [ 10.476554] gitaly[1358]: time="2026-07-21T07:43:27.164Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=1358 gitlab # [ 10.477370] gitaly[1358]: time="2026-07-21T07:43:27.167Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3191043274 clear_duration_ms=0 pid=1358 storage=default gitlab # [ 10.533913] gitaly[1358]: time="2026-07-21T07:43:27.224Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=1358 gitlab # [ 10.536174] gitaly[1358]: time="2026-07-21T07:43:27.224Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=1358 schema=unix gitlab # [ 10.538122] gitaly[1358]: time="2026-07-21T07:43:27.224Z" level=info msg="listening at address" address=/run/gitaly/gitaly-1358/sock.d/intern pid=1358 schema=unix gitlab # [ 10.540806] gitaly[1358]: time="2026-07-21T07:43:27.232Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=1358 gitlab # [ 10.542478] gitaly[1358]: time="2026-07-21T07:43:27.233Z" level=info msg="Started Gitaly" duration_ms=231 pid=1358 gitlab # [ 10.544067] gitaly[1358]: time="2026-07-21T07:43:27.233Z" level=info msg="finished tempdir cleaner walk" pid=1358 storage=default time_ms=0 gitlab # [ 10.545901] gitaly[1358]: time="2026-07-21T07:43:27.233Z" level=info msg="counting repositories" pid=1358 gitlab # [ 10.547333] gitaly[1358]: time="2026-07-21T07:43:27.233Z" level=info msg="starting to count repositories" pid=1358 storage_path=/var/gitlab/state/repositories gitlab # [ 10.549662] gitaly[1358]: time="2026-07-21T07:43:27.233Z" level=info msg="maintenance: daily scheduled" pid=1358 scheduled="2026-07-21 12:00:00 +0000 UTC" gitlab # [ 10.549968] gitaly[1358]: time="2026-07-21T07:43:27.234Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=1358 storage_path=/var/gitlab/state/repositories gitlab # [ 10.550637] gitaly[1358]: time="2026-07-21T07:43:27.234Z" level=info msg="completed counting all repositories" counting_duration=0 pid=1358 gitlab # [ 11.105076] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1369]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.107513] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1369]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.130813] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1369]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.407490] gitaly[1358]: time="2026-07-21T07:43:29.097Z" level=info msg="License database preloaded" duration_ms=2096 pid=1358 gitlab # [ 14.002072] dhcpcd[1235]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 14.004434] dhcpcd[1235]: eth0: adding route to 10.0.2.0/24 gitlab # [ 14.006419] dhcpcd[1235]: eth0: adding default route via 10.0.2.2 gitlab # [ 14.087585] systemd[1]: Started DHCP Client. gitlab # [ 14.090260] systemd[1]: Reached target Network is Online. gitlab # [ 38.788746] gitlab-workhorse[1289]: time="2026-07-21T07:43:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 39.548090] postgres[1720]: [1720] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 39.548939] postgres[1720]: [1720] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 39.549720] postgres[1720]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 39.550721] postgres[1720]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 39.552336] postgres[1720]: attidentity AS identity, gitlab # [ 39.553256] postgres[1720]: attgenerated as attgenerated gitlab # [ 39.553985] postgres[1720]: FROM pg_attribute a gitlab # [ 39.554501] postgres[1720]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 39.555206] postgres[1720]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 39.556158] postgres[1720]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 39.556958] postgres[1720]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 39.557799] postgres[1720]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 39.558638] postgres[1720]: ORDER BY a.attnum gitlab # [ 39.559480] postgres[1720]: gitlab # [ 39.768245] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1369]: W, [2026-07-21T07:43:56.459131 #1369] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 45.067397] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1785]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 45.068240] ibq8wv904zk2qy52df1dc0x5kfbjfac2-gitlab-db-config[1785]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 48.798424] gitlab-workhorse[1289]: time="2026-07-21T07:44:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 58.799599] gitlab-workhorse[1289]: time="2026-07-21T07:44:15Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 58.890474] postgres[1842]: [1842] ERROR: relation "table_without_models" does not exist gitlab # [ 58.891227] postgres[1842]: [1842] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 68.808813] gitlab-workhorse[1289]: time="2026-07-21T07:44:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 70.931141] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 70.934805] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 70.937816] systemd[1]: Starting gitlab.service... gitlab # [ 71.272268] bundle[2009]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 71.273184] bundle[2009]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 71.273908] bundle[2009]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 73.946963] sidekiq-cluster[2008]: {"severity":"INFO","time":"2026-07-21T07:44:30.621Z","message":"Starting cluster with 1 processes"} gitlab # [ 74.043547] bundle[2009]: {"timestamp":"2026-07-21T07:44:30.715Z","pid":2009,"message":"* Enabling systemd notification integration"} gitlab # [ 74.044233] bundle[2009]: {"timestamp":"2026-07-21T07:44:30.734Z","pid":2009,"message":"Puma starting in cluster mode..."} gitlab # [ 74.044910] bundle[2009]: {"timestamp":"2026-07-21T07:44:30.734Z","pid":2009,"message":"* Puma version: 7.2.0 (\"On The Corner\")"} gitlab # [ 74.045771] bundle[2009]: {"timestamp":"2026-07-21T07:44:30.734Z","pid":2009,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 74.046672] bundle[2009]: {"timestamp":"2026-07-21T07:44:30.734Z","pid":2009,"message":"* Min threads: 0"} gitlab # [ 74.047576] bundle[2009]: {"timestamp":"2026-07-21T07:44:30.734Z","pid":2009,"message":"* Max threads: 4"} gitlab # [ 74.048495] bundle[2009]: {"timestamp":"2026-07-21T07:44:30.734Z","pid":2009,"message":"* Environment: production"} gitlab # [ 74.049153] bundle[2009]: {"timestamp":"2026-07-21T07:44:30.734Z","pid":2009,"message":"* Master PID: 2009"} gitlab # [ 74.049804] bundle[2009]: {"timestamp":"2026-07-21T07:44:30.734Z","pid":2009,"message":"* Workers: 2"} gitlab # [ 74.050489] bundle[2009]: {"timestamp":"2026-07-21T07:44:30.734Z","pid":2009,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 74.051170] bundle[2009]: {"timestamp":"2026-07-21T07:44:30.734Z","pid":2009,"message":"* Preloading application"} gitlab # [ 78.818683] gitlab-workhorse[1289]: time="2026-07-21T07:44:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 88.819641] gitlab-workhorse[1289]: time="2026-07-21T07:44:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 96.224070] bundle[2009]: W, [2026-07-21T07:44:52.914617 #2009] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 98.828868] gitlab-workhorse[1289]: time="2026-07-21T07:44:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 98.878815] sidekiq-cluster[2039]: 2026-07-21T07:44:55.569Z pid=2039 tid=u4f INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2039, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>430858240} gitlab # [ 99.053675] sidekiq-cluster[2039]: W, [2026-07-21T07:44:55.744762 #2039] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 99.094082] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:55.784Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 100.923424] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.613Z","message":"GitLab reliable fetch activated!"} gitlab # [ 100.924228] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.613Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 100.955919] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.647Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 100.963384] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.654Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 100.970629] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.661Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 100.978317] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.669Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 100.981308] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.672Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 100.985259] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.676Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 100.989058] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.680Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 100.991938] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.683Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 100.995337] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.686Z","message":"Cron Jobs - added job with name expire_build_artifacts_worker in the namespace default"} gitlab # [ 100.998450] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.689Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 101.001971] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.693Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 101.006188] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.697Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 101.009882] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.701Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 101.013103] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.704Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 101.016437] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.707Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 101.019967] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.711Z","message":"Cron Jobs - added job with name environments_auto_delete_cron_worker in the namespace default"} gitlab # [ 101.023320] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.714Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 101.027543] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.718Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 101.030624] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.721Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 101.033601] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.724Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 101.036534] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.727Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 101.039438] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.730Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 101.042066] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.733Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 101.044647] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.735Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 101.046975] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.738Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 101.049269] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.740Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 101.051784] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.743Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 101.054323] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.745Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 101.056881] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.748Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 101.059734] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.751Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 101.062431] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.753Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 101.064676] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.755Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 101.067323] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.758Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 101.071173] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.762Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 101.075395] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.766Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 101.079072] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.770Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 101.082456] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.773Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 101.087600] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.778Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 101.091087] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.782Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 101.092904] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.784Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 101.094532] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.785Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 101.096127] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.787Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 101.097604] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.788Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 101.099183] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.790Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 101.103234] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.794Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 101.107108] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.798Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 101.111514] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.802Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 101.115091] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.806Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 101.119401] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.810Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 101.123732] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.815Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 101.127876] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.819Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 101.132081] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.823Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 101.135893] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.827Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 101.146196] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.837Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 101.150885] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.842Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 101.155177] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.846Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 101.157463] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.848Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 101.161317] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.852Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 101.164117] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.855Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 101.167527] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.858Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 101.171390] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.862Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 101.176702] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.867Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 101.179125] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.870Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 101.183674] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.874Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 101.190184] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.881Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 101.192998] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.884Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 101.197101] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.888Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 101.201983] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.893Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 101.204607] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.895Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 101.233988] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.925Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 101.236247] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.927Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 101.243167] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.934Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 101.249841] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.941Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 101.257924] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.949Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 101.265206] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.956Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 101.272756] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.963Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 101.277801] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.969Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 101.283920] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.975Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 101.288894] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.980Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 101.294609] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.985Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 101.299211] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.990Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 101.304847] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:57.996Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 101.310693] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.001Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 101.316655] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.007Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 101.322305] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.013Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 101.327303] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.018Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 101.332910] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.024Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 101.338589] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.029Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 101.344450] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.035Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 101.349081] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.040Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 101.354938] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.046Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 101.359642] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.050Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 101.365119] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.056Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 101.369809] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.061Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 101.376799] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.067Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 101.382660] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.073Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 101.388763] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.079Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 101.394197] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.085Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 101.398917] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.090Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 101.418792] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.110Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 101.424807] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.116Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 101.432191] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.123Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 101.440258] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.131Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 101.447949] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.139Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 101.455193] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.146Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 101.464907] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.156Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 101.468715] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.159Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 101.476425] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.167Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 101.485862] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.177Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 101.493134] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:44:58.183Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 108.839693] gitlab-workhorse[1289]: time="2026-07-21T07:45:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 118.849310] gitlab-workhorse[1289]: time="2026-07-21T07:45:15Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 122.070890] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:18.761Z","message":"Booted Rails 7.2.3 application in production environment"} gitlab # [ 122.071537] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:18.761Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 122.072552] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:18.762Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 122.073267] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:18.762Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 122.078737] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:18.769Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 123.395767] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:20.086Z","message":"Cleaning working queues"} gitlab # [ 123.396970] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:20.087Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b106adf6d4716acf5f538889","created_at":"2026-07-21T07:44:27.025Z","trace_propagation_headers":{"sentry-trace":"648607d509e54e38ae19575ce33fa3e3-322da25b96534ba6","baggage":"sentry-trace_id=648607d509e54e38ae19575ce33fa3e3,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"11d319c18c41f6357357db09642aa26e","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-21T07:44:27.032Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b106adf6d4716acf5f538889: start","job_status":"start","queue_duration_s":53.055365,"scheduling_latency_s":53.055438} gitlab # [ 123.591957] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:20.282Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"b106adf6d4716acf5f538889","created_at":"2026-07-21T07:44:27.025Z","trace_propagation_headers":{"sentry-trace":"648607d509e54e38ae19575ce33fa3e3-322da25b96534ba6","baggage":"sentry-trace_id=648607d509e54e38ae19575ce33fa3e3,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"11d319c18c41f6357357db09642aa26e","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-21T07:44:27.032Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-b106adf6d4716acf5f538889: done: 0.194929 sec","job_status":"done","queue_duration_s":53.055365,"scheduling_latency_s":53.055438,"redis_calls":31,"redis_duration_s":0.008215,"redis_read_bytes":284,"redis_write_bytes":3897,"redis_feature_flag_calls":20,"redis_feature_flag_duration_s":0.004833,"redis_feature_flag_read_bytes":274,"redis_feature_flag_write_bytes":2650,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002092,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00129,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":16,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.096267,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.08851376400000001,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.194929,"completed_at":"2026-07-21T07:45:20.282Z","load_balancing_strategy":"primary","db_duration_s":0.049946,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 123.870689] bundle[2009]: {"timestamp":"2026-07-21T07:45:20.561Z","pid":2009,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 123.871088] bundle[2009]: {"timestamp":"2026-07-21T07:45:20.561Z","pid":2009,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 123.872261] bundle[2009]: {"timestamp":"2026-07-21T07:45:20.561Z","pid":2009,"message":"! #\u003cThread:0x0000712913e26a00 /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/g5qraa51hgg5fl7n3da89ygf00qfv95x-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 123.873364] bundle[2009]: {"timestamp":"2026-07-21T07:45:20.561Z","pid":2009,"message":"! #\u003cThread:0x000071291405c590 /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/wg5r0zmy3rh1kga7z4z7mbs298p0wxli-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 123.874539] bundle[2009]: {"timestamp":"2026-07-21T07:45:20.561Z","pid":2009,"message":"Use Ctrl-C to stop"} gitlab # [ 123.876053] bundle[2009]: {"timestamp":"2026-07-21T07:45:20.561Z","pid":2009,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 124.235130] bundle[2009]: {"timestamp":"2026-07-21T07:45:20.925Z","pid":2009,"message":"- Worker 0 (PID: 2538) booted in 0.33s, phase: 0"} gitlab # [ 124.235617] bundle[2009]: {"timestamp":"2026-07-21T07:45:20.926Z","pid":2009,"message":"- Worker 1 (PID: 2542) booted in 0.31s, phase: 0"} gitlab # [ 124.244099] systemd[1]: Started gitlab.service. gitlab # [ 124.246251] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 124.251263] systemd[1]: Starting setup-gitlab.service... gitlab # [ 124.274964] setup-gitlab-start[2566]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 126.609703] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:45:23 +0000] "GET /users/sign_in HTTP/1.1" 200 10624 "" "curl/8.20.0" 2309 gitlab # [ 126.616026] setup-gitlab-start[2585]: ++ curl -fs -X POST -H 'Content-Type: application/json' -d '{"grant_type":"password","username":"root","password":"notproduction"}' http://gitlab/oauth/token gitlab # [ 126.618376] setup-gitlab-start[2586]: ++ jq -r .access_token gitlab # [ 127.890462] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:45:24 +0000] "POST /oauth/token HTTP/1.1" 200 244 "" "curl/8.20.0" 1258 gitlab # [ 127.896708] setup-gitlab-start[2566]: + BEARER=41c843e4023b9531ffbba8bafe664da433f9ff860e7ff8e30ee758b488c40e28 gitlab # [ 127.899853] setup-gitlab-start[2566]: + AUTH='Authorization: Bearer 41c843e4023b9531ffbba8bafe664da433f9ff860e7ff8e30ee758b488c40e28' gitlab # [ 127.903552] setup-gitlab-start[2566]: + curl -fs -X PUT -H 'Authorization: Bearer 41c843e4023b9531ffbba8bafe664da433f9ff860e7ff8e30ee758b488c40e28' 'http://gitlab/api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true' gitlab # [ 128.636023] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:45:25 +0000] "PUT /api/v4/application/settings?allow_local_requests_from_web_hooks_and_services=true HTTP/1.1" 200 17133 "" "curl/8.20.0" 726 gitlab # [ 128.644021] setup-gitlab-start[2566]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner Rails.cache.clear gitlab # [ 128.658160] runuser[2608]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 129.053158] setup-gitlab-start[2609]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 129.053964] setup-gitlab-start[2609]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 129.058098] setup-gitlab-start[2609]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 129.594761] setup-gitlab-start[2609]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 129.597917] setup-gitlab-start[2609]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 129.602864] setup-gitlab-start[2609]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 135.247502] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:31.937Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c0c15c5167e0dadb7c55d906","created_at":"2026-07-21T07:45:31.906Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:31.935Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c0c15c5167e0dadb7c55d906: start","job_status":"start","queue_duration_s":0.002127,"scheduling_latency_s":0.002161} gitlab # [ 135.522758] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:32.213Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c0c15c5167e0dadb7c55d906","created_at":"2026-07-21T07:45:31.906Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:31.935Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c0c15c5167e0dadb7c55d906: done: 0.275602 sec","job_status":"done","queue_duration_s":0.002127,"scheduling_latency_s":0.002161,"redis_calls":71,"redis_duration_s":0.100482,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.003662,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001215,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.092115,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00349,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.11835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001594058,"duration_s":0.275602,"completed_at":"2026-07-21T07:45:32.213Z","load_balancing_strategy":"primary","db_duration_s":0.01438,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 135.580794] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:32.271Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"fcd360ebbfb46a60fdc2f440","created_at":"2026-07-21T07:45:31.944Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad29680c792a06b99729842c30579a99","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:31.959Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-fcd360ebbfb46a60fdc2f440: start","job_status":"start","queue_duration_s":0.311744,"scheduling_latency_s":0.311777} gitlab # [ 135.684124] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:32.374Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"fcd360ebbfb46a60fdc2f440","created_at":"2026-07-21T07:45:31.944Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad29680c792a06b99729842c30579a99","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:31.959Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-fcd360ebbfb46a60fdc2f440: done: 0.103038 sec","job_status":"done","queue_duration_s":0.311744,"scheduling_latency_s":0.311777,"redis_calls":13,"redis_duration_s":0.015182000000000001,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002489,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.006142,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006551,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.030037058000000002,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.103038,"completed_at":"2026-07-21T07:45:32.374Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02007,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 135.748566] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:32.388Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"381a10c9c453b6e4a1d64c26","created_at":"2026-07-21T07:45:31.978Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee5292e958405c6b79dbd22af54fd3e0","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.022Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-381a10c9c453b6e4a1d64c26: start","job_status":"start","queue_duration_s":0.366496,"scheduling_latency_s":0.366531} gitlab # [ 136.117347] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:32.808Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"381a10c9c453b6e4a1d64c26","created_at":"2026-07-21T07:45:31.978Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee5292e958405c6b79dbd22af54fd3e0","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.022Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-381a10c9c453b6e4a1d64c26: done: 0.419454 sec","job_status":"done","queue_duration_s":0.366496,"scheduling_latency_s":0.366531,"redis_calls":574,"redis_duration_s":0.20508700000000002,"redis_read_bytes":573,"redis_write_bytes":49866,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000627,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00152,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.20294,"redis_shared_state_read_bytes":567,"redis_shared_state_write_bytes":49017,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.236762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000458159,"duration_s":0.419454,"completed_at":"2026-07-21T07:45:32.808Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000738,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 136.166520] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:32.857Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"f278521bf62eab811534765e","created_at":"2026-07-21T07:45:32.034Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0eca3c364cb4c9b3fe959cf8c9b1bb4","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.045Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-f278521bf62eab811534765e: start","job_status":"start","queue_duration_s":0.811667,"scheduling_latency_s":0.811691} gitlab # [ 136.203728] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:32.894Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Issues::RescheduleStuckIssueRebalancesWorker","jid":"f278521bf62eab811534765e","created_at":"2026-07-21T07:45:32.034Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0eca3c364cb4c9b3fe959cf8c9b1bb4","meta.root_caller_id":"Cronjob","meta.feature_category":"team_planning","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fc2fa70b35c572764e1a8fbcbc4fdd3df48ec5a5855870d65915560d23915134","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.045Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Issues::RescheduleStuckIssueRebalancesWorker JID-f278521bf62eab811534765e: done: 0.036824 sec","job_status":"done","queue_duration_s":0.811667,"scheduling_latency_s":0.811691,"redis_calls":11,"redis_duration_s":0.002992,"redis_read_bytes":9,"redis_write_bytes":1239,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000434,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001439,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001119,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":298,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015154,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000357308,"duration_s":0.036824,"completed_at":"2026-07-21T07:45:32.894Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000541,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 136.241598] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:32.896Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"797201e53ba8eec7c0009e99","created_at":"2026-07-21T07:45:32.059Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3b1f7fc1bb9ede9a2ce2c30ac7fb63d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.070Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-797201e53ba8eec7c0009e99: start","job_status":"start","queue_duration_s":0.826006,"scheduling_latency_s":0.826036} gitlab # [ 136.400855] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.090Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"797201e53ba8eec7c0009e99","created_at":"2026-07-21T07:45:32.059Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3b1f7fc1bb9ede9a2ce2c30ac7fb63d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.070Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-797201e53ba8eec7c0009e99: done: 0.194296 sec","job_status":"done","queue_duration_s":0.826006,"scheduling_latency_s":0.826036,"redis_calls":14,"redis_duration_s":0.004137,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001413,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001461,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001263,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.161139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000812394,"exclusive_lock_hold_duration_s":0.16641863699999998,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.194296,"completed_at":"2026-07-21T07:45:33.090Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020745,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 136.402270] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.092Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"5f5c8a881ca6bbded87d8caf","created_at":"2026-07-21T07:45:32.083Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9074c800357ef8c8cd18dae0cf592ca7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.096Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-5f5c8a881ca6bbded87d8caf: start","job_status":"start","queue_duration_s":0.996047,"scheduling_latency_s":0.996064} gitlab # [ 136.430440] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.117Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"5f5c8a881ca6bbded87d8caf","created_at":"2026-07-21T07:45:32.083Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9074c800357ef8c8cd18dae0cf592ca7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.096Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-5f5c8a881ca6bbded87d8caf: done: 0.025511 sec","job_status":"done","queue_duration_s":0.996047,"scheduling_latency_s":0.996064,"redis_calls":19,"redis_duration_s":0.002548,"redis_read_bytes":22,"redis_write_bytes":4781,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000538,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000195,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001337,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000478,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00020254,"duration_s":0.025511,"completed_at":"2026-07-21T07:45:33.117Z","load_balancing_strategy":"primary","db_duration_s":0.004457,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 136.493574] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.118Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"3f0b71ee2bdfb8d0cddb6357","created_at":"2026-07-21T07:45:32.107Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79b72151724da27dd1f641b77af14a59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.119Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-3f0b71ee2bdfb8d0cddb6357: start","job_status":"start","queue_duration_s":0.999044,"scheduling_latency_s":0.999059} gitlab # [ 136.502818] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.126Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 136.543826] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.130Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"3f0b71ee2bdfb8d0cddb6357","created_at":"2026-07-21T07:45:32.107Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79b72151724da27dd1f641b77af14a59","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.119Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-3f0b71ee2bdfb8d0cddb6357: done: 0.012352 sec","job_status":"done","queue_duration_s":0.999044,"scheduling_latency_s":0.999059,"redis_calls":10,"redis_duration_s":0.001326,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000247,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000817,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000262,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000200305,"duration_s":0.012352,"completed_at":"2026-07-21T07:45:33.130Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000729,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 136.545034] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.131Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"5373bcc1730d9c0486904fe1","created_at":"2026-07-21T07:45:32.139Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1199fd1335d876736d6c2d3ef8a1baf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.150Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-5373bcc1730d9c0486904fe1: start","job_status":"start","queue_duration_s":0.981745,"scheduling_latency_s":0.981758} gitlab # [ 136.557674] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.139Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 136.561025] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.143Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"5373bcc1730d9c0486904fe1","created_at":"2026-07-21T07:45:32.139Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1199fd1335d876736d6c2d3ef8a1baf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.150Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-5373bcc1730d9c0486904fe1: done: 0.011927 sec","job_status":"done","queue_duration_s":0.981745,"scheduling_latency_s":0.981758,"redis_calls":10,"redis_duration_s":0.001212,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000271,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000693,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000191644,"duration_s":0.011927,"completed_at":"2026-07-21T07:45:33.143Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000237,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 136.607505] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.145Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"e616a767af5ad22d94f1770e","created_at":"2026-07-21T07:45:32.159Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed23afeb0601d17de5a76673019a15a7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.170Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-e616a767af5ad22d94f1770e: start","job_status":"start","queue_duration_s":0.974712,"scheduling_latency_s":0.974726} gitlab # [ 136.612958] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.172Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"e616a767af5ad22d94f1770e","created_at":"2026-07-21T07:45:32.159Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed23afeb0601d17de5a76673019a15a7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.170Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-e616a767af5ad22d94f1770e: done: 0.027688 sec","job_status":"done","queue_duration_s":0.974712,"scheduling_latency_s":0.974726,"redis_calls":12,"redis_duration_s":0.001616,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000248,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000758,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00061,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000391949,"exclusive_lock_hold_duration_s":0.007999899,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.027688,"completed_at":"2026-07-21T07:45:33.172Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009231,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 136.654667] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9243611665b6540a69fe9670","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9243611665b6540a69fe9670: start","job_status":"start","queue_duration_s":0.976897,"scheduling_latency_s":0.976912} gitlab # [ 136.666582] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9243611665b6540a69fe9670","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9243611665b6540a69fe9670: done: 0.014645 sec","job_status":"done","queue_duration_s":0.976897,"scheduling_latency_s":0.976912,"redis_calls":16,"redis_duration_s":0.002067,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000251,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000681,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001135,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00019807,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014645,"completed_at":"2026-07-21T07:45:33.188Z","load_balancing_strategy":"primary","db_duration_s":0.000256,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 136.728808] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a23da7f232f9a09aa6f0687","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a23da7f232f9a09aa6f0687: start","job_status":"start","queue_duration_s":0.992592,"scheduling_latency_s":0.992605} gitlab # [ 136.729721] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a23da7f232f9a09aa6f0687","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a23da7f232f9a09aa6f0687: done: 0.010456 sec","job_status":"done","queue_duration_s":0.992592,"scheduling_latency_s":0.992605,"redis_calls":13,"redis_duration_s":0.001645,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000588,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010456,"completed_at":"2026-07-21T07:45:33.199Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 136.775696] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0dadf1ca2da1e5a4508db96e","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0dadf1ca2da1e5a4508db96e: start","job_status":"start","queue_duration_s":1.004334,"scheduling_latency_s":1.00435} gitlab # [ 136.776607] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.211Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0dadf1ca2da1e5a4508db96e","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0dadf1ca2da1e5a4508db96e: done: 0.010701 sec","job_status":"done","queue_duration_s":1.004334,"scheduling_latency_s":1.00435,"redis_calls":13,"redis_duration_s":0.0014960000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010701,"completed_at":"2026-07-21T07:45:33.211Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 136.836786] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.212Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ce4e116746c7911137a2b12","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ce4e116746c7911137a2b12: start","job_status":"start","queue_duration_s":1.015949,"scheduling_latency_s":1.015964} gitlab # [ 136.837781] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ce4e116746c7911137a2b12","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ce4e116746c7911137a2b12: done: 0.010329 sec","job_status":"done","queue_duration_s":1.015949,"scheduling_latency_s":1.015964,"redis_calls":13,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010329,"completed_at":"2026-07-21T07:45:33.223Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 136.845885] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84b7470802c504657fc54ce5","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84b7470802c504657fc54ce5: start","job_status":"start","queue_duration_s":1.027281,"scheduling_latency_s":1.027295} gitlab # [ 136.891737] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84b7470802c504657fc54ce5","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84b7470802c504657fc54ce5: done: 0.01192 sec","job_status":"done","queue_duration_s":1.027281,"scheduling_latency_s":1.027295,"redis_calls":13,"redis_duration_s":0.0018579999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001298,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01192,"completed_at":"2026-07-21T07:45:33.236Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 136.892720] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b311f510ca598cb6374cc06","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b311f510ca598cb6374cc06: start","job_status":"start","queue_duration_s":1.040289,"scheduling_latency_s":1.040309} gitlab # [ 136.901061] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b311f510ca598cb6374cc06","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b311f510ca598cb6374cc06: done: 0.009855 sec","job_status":"done","queue_duration_s":1.040289,"scheduling_latency_s":1.040309,"redis_calls":13,"redis_duration_s":0.001634,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009855,"completed_at":"2026-07-21T07:45:33.246Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 136.939663] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"380966198b58468a06683051","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-380966198b58468a06683051: start","job_status":"start","queue_duration_s":1.050936,"scheduling_latency_s":1.050948} gitlab # [ 136.945696] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"380966198b58468a06683051","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-380966198b58468a06683051: done: 0.009001 sec","job_status":"done","queue_duration_s":1.050936,"scheduling_latency_s":1.050948,"redis_calls":13,"redis_duration_s":0.001714,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001198,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009001,"completed_at":"2026-07-21T07:45:33.256Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.008330] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"924b8dd7cb1be89949d7eaa3","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-924b8dd7cb1be89949d7eaa3: start","job_status":"start","queue_duration_s":1.060744,"scheduling_latency_s":1.060757} gitlab # [ 137.008929] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.266Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"924b8dd7cb1be89949d7eaa3","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-924b8dd7cb1be89949d7eaa3: done: 0.008949 sec","job_status":"done","queue_duration_s":1.060744,"scheduling_latency_s":1.060757,"redis_calls":13,"redis_duration_s":0.00157,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008949,"completed_at":"2026-07-21T07:45:33.266Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.024723] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e2010884b488e4966d9e737","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e2010884b488e4966d9e737: start","job_status":"start","queue_duration_s":1.070475,"scheduling_latency_s":1.070487} gitlab # [ 137.066869] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.276Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1e2010884b488e4966d9e737","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1e2010884b488e4966d9e737: done: 0.008981 sec","job_status":"done","queue_duration_s":1.070475,"scheduling_latency_s":1.070487,"redis_calls":13,"redis_duration_s":0.0015539999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005655,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008981,"completed_at":"2026-07-21T07:45:33.276Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.105688] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.277Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6584b3e930a512bc6a94141","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6584b3e930a512bc6a94141: start","job_status":"start","queue_duration_s":1.080267,"scheduling_latency_s":1.08028} gitlab # [ 137.106598] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6584b3e930a512bc6a94141","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6584b3e930a512bc6a94141: done: 0.008877 sec","job_status":"done","queue_duration_s":1.080267,"scheduling_latency_s":1.08028,"redis_calls":13,"redis_duration_s":0.0016189999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008877,"completed_at":"2026-07-21T07:45:33.285Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.114588] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75ee75b6e30d82a296a11fb5","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75ee75b6e30d82a296a11fb5: start","job_status":"start","queue_duration_s":1.090003,"scheduling_latency_s":1.090016} gitlab # [ 137.125639] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.295Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75ee75b6e30d82a296a11fb5","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75ee75b6e30d82a296a11fb5: done: 0.00887 sec","job_status":"done","queue_duration_s":1.090003,"scheduling_latency_s":1.090016,"redis_calls":13,"redis_duration_s":0.001571,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00887,"completed_at":"2026-07-21T07:45:33.295Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.168694] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51fb3ec9a2d9329941b1d807","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51fb3ec9a2d9329941b1d807: start","job_status":"start","queue_duration_s":1.099729,"scheduling_latency_s":1.099742} gitlab # [ 137.170028] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51fb3ec9a2d9329941b1d807","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51fb3ec9a2d9329941b1d807: done: 0.009275 sec","job_status":"done","queue_duration_s":1.099729,"scheduling_latency_s":1.099742,"redis_calls":13,"redis_duration_s":0.001608,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001074,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009275,"completed_at":"2026-07-21T07:45:33.305Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.208040] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.306Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f54534528d19860fbac5d2c0","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f54534528d19860fbac5d2c0: start","job_status":"start","queue_duration_s":1.1099,"scheduling_latency_s":1.109914} gitlab # [ 137.214671] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f54534528d19860fbac5d2c0","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f54534528d19860fbac5d2c0: done: 0.00889 sec","job_status":"done","queue_duration_s":1.1099,"scheduling_latency_s":1.109914,"redis_calls":13,"redis_duration_s":0.001591,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00889,"completed_at":"2026-07-21T07:45:33.315Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.249743] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6798fcb9fcef7b0727d866a8","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6798fcb9fcef7b0727d866a8: start","job_status":"start","queue_duration_s":1.119647,"scheduling_latency_s":1.11966} gitlab # [ 137.287729] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.325Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6798fcb9fcef7b0727d866a8","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6798fcb9fcef7b0727d866a8: done: 0.008928 sec","job_status":"done","queue_duration_s":1.119647,"scheduling_latency_s":1.11966,"redis_calls":13,"redis_duration_s":0.0015340000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008928,"completed_at":"2026-07-21T07:45:33.325Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.294237] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"718119804a773b57bbfc80e6","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-718119804a773b57bbfc80e6: start","job_status":"start","queue_duration_s":1.129416,"scheduling_latency_s":1.129429} gitlab # [ 137.337248] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"718119804a773b57bbfc80e6","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-718119804a773b57bbfc80e6: done: 0.00883 sec","job_status":"done","queue_duration_s":1.129416,"scheduling_latency_s":1.129429,"redis_calls":13,"redis_duration_s":0.001573,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001063,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00883,"completed_at":"2026-07-21T07:45:33.335Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.338606] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b7c8c942bd8cdb31af928f4","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b7c8c942bd8cdb31af928f4: start","job_status":"start","queue_duration_s":1.139041,"scheduling_latency_s":1.139054} gitlab # [ 137.349618] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b7c8c942bd8cdb31af928f4","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b7c8c942bd8cdb31af928f4: done: 0.008899 sec","job_status":"done","queue_duration_s":1.139041,"scheduling_latency_s":1.139054,"redis_calls":13,"redis_duration_s":0.0016289999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008899,"completed_at":"2026-07-21T07:45:33.344Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.404330] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32a0bc343c8e6917776e0370","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32a0bc343c8e6917776e0370: start","job_status":"start","queue_duration_s":1.148809,"scheduling_latency_s":1.148822} gitlab # [ 137.404957] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32a0bc343c8e6917776e0370","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32a0bc343c8e6917776e0370: done: 0.008885 sec","job_status":"done","queue_duration_s":1.148809,"scheduling_latency_s":1.148822,"redis_calls":13,"redis_duration_s":0.0016129999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008885,"completed_at":"2026-07-21T07:45:33.354Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.446075] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c21c84a6316bbea5bedeab44","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c21c84a6316bbea5bedeab44: start","job_status":"start","queue_duration_s":1.158469,"scheduling_latency_s":1.158482} gitlab # [ 137.446810] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c21c84a6316bbea5bedeab44","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c21c84a6316bbea5bedeab44: done: 0.008908 sec","job_status":"done","queue_duration_s":1.158469,"scheduling_latency_s":1.158482,"redis_calls":13,"redis_duration_s":0.001614,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008908,"completed_at":"2026-07-21T07:45:33.364Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.497557] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee8e3fdf1647089f02f283d5","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee8e3fdf1647089f02f283d5: start","job_status":"start","queue_duration_s":1.168215,"scheduling_latency_s":1.168227} gitlab # [ 137.498521] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee8e3fdf1647089f02f283d5","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee8e3fdf1647089f02f283d5: done: 0.011445 sec","job_status":"done","queue_duration_s":1.168215,"scheduling_latency_s":1.168227,"redis_calls":13,"redis_duration_s":0.003916,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003373,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011445,"completed_at":"2026-07-21T07:45:33.376Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.538720] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4be5600084fc729c94620c41","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4be5600084fc729c94620c41: start","job_status":"start","queue_duration_s":1.180454,"scheduling_latency_s":1.180468} gitlab # [ 137.539321] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4be5600084fc729c94620c41","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4be5600084fc729c94620c41: done: 0.008959 sec","job_status":"done","queue_duration_s":1.180454,"scheduling_latency_s":1.180468,"redis_calls":13,"redis_duration_s":0.001695,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001139,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005587,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008959,"completed_at":"2026-07-21T07:45:33.386Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.580068] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85a80cf8c108f93fa121deb9","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85a80cf8c108f93fa121deb9: start","job_status":"start","queue_duration_s":1.190287,"scheduling_latency_s":1.190299} gitlab # [ 137.580714] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.396Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85a80cf8c108f93fa121deb9","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85a80cf8c108f93fa121deb9: done: 0.008928 sec","job_status":"done","queue_duration_s":1.190287,"scheduling_latency_s":1.190299,"redis_calls":13,"redis_duration_s":0.00159,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008928,"completed_at":"2026-07-21T07:45:33.396Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.624779] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.396Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3df4c4904f6e741c10848f4b","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3df4c4904f6e741c10848f4b: start","job_status":"start","queue_duration_s":1.20002,"scheduling_latency_s":1.200033} gitlab # [ 137.625679] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3df4c4904f6e741c10848f4b","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3df4c4904f6e741c10848f4b: done: 0.010806 sec","job_status":"done","queue_duration_s":1.20002,"scheduling_latency_s":1.200033,"redis_calls":13,"redis_duration_s":0.00183,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000643,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001187,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010806,"completed_at":"2026-07-21T07:45:33.407Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.682530] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"014f83f6215411ead12adcc1","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-014f83f6215411ead12adcc1: start","job_status":"start","queue_duration_s":1.212003,"scheduling_latency_s":1.212022} gitlab # [ 137.683249] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"014f83f6215411ead12adcc1","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-014f83f6215411ead12adcc1: done: 0.010033 sec","job_status":"done","queue_duration_s":1.212003,"scheduling_latency_s":1.212022,"redis_calls":13,"redis_duration_s":0.001489,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001005,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010033,"completed_at":"2026-07-21T07:45:33.418Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.729506] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8f51330b980ba7941fd4319","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8f51330b980ba7941fd4319: start","job_status":"start","queue_duration_s":1.223091,"scheduling_latency_s":1.223109} gitlab # [ 137.730223] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8f51330b980ba7941fd4319","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8f51330b980ba7941fd4319: done: 0.00961 sec","job_status":"done","queue_duration_s":1.223091,"scheduling_latency_s":1.223109,"redis_calls":13,"redis_duration_s":0.0015569999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00961,"completed_at":"2026-07-21T07:45:33.429Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.772541] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b8604e3a5835b90d3fb975f","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b8604e3a5835b90d3fb975f: start","job_status":"start","queue_duration_s":1.233567,"scheduling_latency_s":1.23358} gitlab # [ 137.773180] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b8604e3a5835b90d3fb975f","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b8604e3a5835b90d3fb975f: done: 0.008892 sec","job_status":"done","queue_duration_s":1.233567,"scheduling_latency_s":1.23358,"redis_calls":13,"redis_duration_s":0.001556,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00569,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008892,"completed_at":"2026-07-21T07:45:33.439Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.814287] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.440Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29e859b28755787479ba4d88","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29e859b28755787479ba4d88: start","job_status":"start","queue_duration_s":1.24335,"scheduling_latency_s":1.243362} gitlab # [ 137.815025] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29e859b28755787479ba4d88","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29e859b28755787479ba4d88: done: 0.008825 sec","job_status":"done","queue_duration_s":1.24335,"scheduling_latency_s":1.243362,"redis_calls":13,"redis_duration_s":0.0015509999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008825,"completed_at":"2026-07-21T07:45:33.449Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.856731] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b033f1fddf15eec593769f4","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b033f1fddf15eec593769f4: start","job_status":"start","queue_duration_s":1.25305,"scheduling_latency_s":1.253063} gitlab # [ 137.857511] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.458Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b033f1fddf15eec593769f4","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b033f1fddf15eec593769f4: done: 0.008839 sec","job_status":"done","queue_duration_s":1.25305,"scheduling_latency_s":1.253063,"redis_calls":13,"redis_duration_s":0.001577,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008839,"completed_at":"2026-07-21T07:45:33.458Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.875252] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.459Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc05eb1e84de498c829b40e5","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc05eb1e84de498c829b40e5: start","job_status":"start","queue_duration_s":1.262677,"scheduling_latency_s":1.262689} gitlab # [ 137.917522] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc05eb1e84de498c829b40e5","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc05eb1e84de498c829b40e5: done: 0.010281 sec","job_status":"done","queue_duration_s":1.262677,"scheduling_latency_s":1.262689,"redis_calls":13,"redis_duration_s":0.001728,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010281,"completed_at":"2026-07-21T07:45:33.469Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.919977] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.470Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b1dd8cba5190c1bc96100bd","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b1dd8cba5190c1bc96100bd: start","job_status":"start","queue_duration_s":1.273765,"scheduling_latency_s":1.273779} gitlab # [ 137.962839] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b1dd8cba5190c1bc96100bd","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b1dd8cba5190c1bc96100bd: done: 0.009037 sec","job_status":"done","queue_duration_s":1.273765,"scheduling_latency_s":1.273779,"redis_calls":13,"redis_duration_s":0.001598,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009037,"completed_at":"2026-07-21T07:45:33.479Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 137.964917] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5fc4a623ed89772da382cce","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5fc4a623ed89772da382cce: start","job_status":"start","queue_duration_s":1.283595,"scheduling_latency_s":1.283607} gitlab # [ 137.976026] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.489Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5fc4a623ed89772da382cce","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5fc4a623ed89772da382cce: done: 0.008884 sec","job_status":"done","queue_duration_s":1.283595,"scheduling_latency_s":1.283607,"redis_calls":13,"redis_duration_s":0.001535,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008884,"completed_at":"2026-07-21T07:45:33.489Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.026652] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b68fd2419c3bfe05d1900165","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b68fd2419c3bfe05d1900165: start","job_status":"start","queue_duration_s":1.29336,"scheduling_latency_s":1.293374} gitlab # [ 138.027607] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.499Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b68fd2419c3bfe05d1900165","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b68fd2419c3bfe05d1900165: done: 0.008933 sec","job_status":"done","queue_duration_s":1.29336,"scheduling_latency_s":1.293374,"redis_calls":13,"redis_duration_s":0.001535,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001039,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008933,"completed_at":"2026-07-21T07:45:33.499Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.077491] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.500Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80e039f6934b22b8c0344daf","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80e039f6934b22b8c0344daf: start","job_status":"start","queue_duration_s":1.30314,"scheduling_latency_s":1.303152} gitlab # [ 138.078121] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.509Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80e039f6934b22b8c0344daf","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80e039f6934b22b8c0344daf: done: 0.00924 sec","job_status":"done","queue_duration_s":1.30314,"scheduling_latency_s":1.303152,"redis_calls":13,"redis_duration_s":0.00156,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00924,"completed_at":"2026-07-21T07:45:33.509Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.131661] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:33.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1aca91d3a256b5432e17dc2","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1aca91d3a256b5432e17dc2: start","job_status":"start","queue_duration_s":1.313181,"scheduling_latency_s":1.313194} gitlab # [ 138.288446] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 1878100023 wd_nsec: 1878100334 gitlab # [ 138.156547] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1aca91d3a256b5432e17dc2","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1aca91d3a256b5432e17dc2: done: 1.337339 sec","job_status":"done","queue_duration_s":1.313181,"scheduling_latency_s":1.313194,"redis_calls":13,"redis_duration_s":0.002143,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001129,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":1.337339,"completed_at":"2026-07-21T07:45:34.847Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.157545] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be32349c149b88767ca011fc","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be32349c149b88767ca011fc: start","job_status":"start","queue_duration_s":2.651873,"scheduling_latency_s":2.651892} gitlab # [ 138.167883] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be32349c149b88767ca011fc","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be32349c149b88767ca011fc: done: 0.010254 sec","job_status":"done","queue_duration_s":2.651873,"scheduling_latency_s":2.651892,"redis_calls":13,"redis_duration_s":0.00166,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001137,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010254,"completed_at":"2026-07-21T07:45:34.858Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.229124] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"546b63b7cc74f470c09bc49f","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-546b63b7cc74f470c09bc49f: start","job_status":"start","queue_duration_s":2.662969,"scheduling_latency_s":2.662981} gitlab # [ 138.229842] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"546b63b7cc74f470c09bc49f","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-546b63b7cc74f470c09bc49f: done: 0.009005 sec","job_status":"done","queue_duration_s":2.662969,"scheduling_latency_s":2.662981,"redis_calls":13,"redis_duration_s":0.001729,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001149,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009005,"completed_at":"2026-07-21T07:45:34.868Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.278720] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2d903a888703d072e9633a6","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2d903a888703d072e9633a6: start","job_status":"start","queue_duration_s":2.672872,"scheduling_latency_s":2.672886} gitlab # [ 138.279557] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2d903a888703d072e9633a6","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2d903a888703d072e9633a6: done: 0.008977 sec","job_status":"done","queue_duration_s":2.672872,"scheduling_latency_s":2.672886,"redis_calls":13,"redis_duration_s":0.00162,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008977,"completed_at":"2026-07-21T07:45:34.878Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.292097] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5f005f0cb4d03d531daf904","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5f005f0cb4d03d531daf904: start","job_status":"start","queue_duration_s":2.682715,"scheduling_latency_s":2.682728} gitlab # [ 138.333124] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5f005f0cb4d03d531daf904","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5f005f0cb4d03d531daf904: done: 0.008976 sec","job_status":"done","queue_duration_s":2.682715,"scheduling_latency_s":2.682728,"redis_calls":13,"redis_duration_s":0.001613,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008976,"completed_at":"2026-07-21T07:45:34.888Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.338901] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2b86fd3f914c535189c3bac","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2b86fd3f914c535189c3bac: start","job_status":"start","queue_duration_s":2.692474,"scheduling_latency_s":2.692486} gitlab # [ 138.350287] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2b86fd3f914c535189c3bac","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2b86fd3f914c535189c3bac: done: 0.008883 sec","job_status":"done","queue_duration_s":2.692474,"scheduling_latency_s":2.692486,"redis_calls":13,"redis_duration_s":0.0015539999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008883,"completed_at":"2026-07-21T07:45:34.898Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.399723] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9df5b2b117e17b8318d5f01","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9df5b2b117e17b8318d5f01: start","job_status":"start","queue_duration_s":2.702219,"scheduling_latency_s":2.702232} gitlab # [ 138.400488] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9df5b2b117e17b8318d5f01","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9df5b2b117e17b8318d5f01: done: 0.009741 sec","job_status":"done","queue_duration_s":2.702219,"scheduling_latency_s":2.702232,"redis_calls":13,"redis_duration_s":0.001826,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001272,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009741,"completed_at":"2026-07-21T07:45:34.908Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.440252] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.909Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0617b75ca20c5d1b0d4aab6","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0617b75ca20c5d1b0d4aab6: start","job_status":"start","queue_duration_s":2.712801,"scheduling_latency_s":2.712813} gitlab # [ 138.441846] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.919Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0617b75ca20c5d1b0d4aab6","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0617b75ca20c5d1b0d4aab6: done: 0.009558 sec","job_status":"done","queue_duration_s":2.712801,"scheduling_latency_s":2.712813,"redis_calls":13,"redis_duration_s":0.001794,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001245,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009558,"completed_at":"2026-07-21T07:45:34.919Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.492643] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85c4037c0677af396d17a8e3","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85c4037c0677af396d17a8e3: start","job_status":"start","queue_duration_s":2.723162,"scheduling_latency_s":2.723175} gitlab # [ 138.493216] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85c4037c0677af396d17a8e3","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85c4037c0677af396d17a8e3: done: 0.009464 sec","job_status":"done","queue_duration_s":2.723162,"scheduling_latency_s":2.723175,"redis_calls":13,"redis_duration_s":0.0016289999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009464,"completed_at":"2026-07-21T07:45:34.929Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.535966] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ee219dc100f42d38f2e4e90","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ee219dc100f42d38f2e4e90: start","job_status":"start","queue_duration_s":2.733401,"scheduling_latency_s":2.733414} gitlab # [ 138.536650] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ee219dc100f42d38f2e4e90","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ee219dc100f42d38f2e4e90: done: 0.009524 sec","job_status":"done","queue_duration_s":2.733401,"scheduling_latency_s":2.733414,"redis_calls":13,"redis_duration_s":0.00159,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009524,"completed_at":"2026-07-21T07:45:34.939Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.602494] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8257929c4e85d8cc8c662b8","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8257929c4e85d8cc8c662b8: start","job_status":"start","queue_duration_s":2.743752,"scheduling_latency_s":2.743765} gitlab # [ 138.603221] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8257929c4e85d8cc8c662b8","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8257929c4e85d8cc8c662b8: done: 0.00961 sec","job_status":"done","queue_duration_s":2.743752,"scheduling_latency_s":2.743765,"redis_calls":13,"redis_duration_s":0.001771,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001265,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00961,"completed_at":"2026-07-21T07:45:34.950Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.650280] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"424ab26f9ebe16119bbfe9bb","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-424ab26f9ebe16119bbfe9bb: start","job_status":"start","queue_duration_s":2.754313,"scheduling_latency_s":2.754327} gitlab # [ 138.650900] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"424ab26f9ebe16119bbfe9bb","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-424ab26f9ebe16119bbfe9bb: done: 0.009409 sec","job_status":"done","queue_duration_s":2.754313,"scheduling_latency_s":2.754327,"redis_calls":13,"redis_duration_s":0.001537,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009409,"completed_at":"2026-07-21T07:45:34.960Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.658900] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c87cae4cc0524be67d30c47","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c87cae4cc0524be67d30c47: start","job_status":"start","queue_duration_s":2.764666,"scheduling_latency_s":2.764679} gitlab # [ 138.670109] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.971Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c87cae4cc0524be67d30c47","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c87cae4cc0524be67d30c47: done: 0.009598 sec","job_status":"done","queue_duration_s":2.764666,"scheduling_latency_s":2.764679,"redis_calls":13,"redis_duration_s":0.0015899999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009598,"completed_at":"2026-07-21T07:45:34.971Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.705580] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e06dbdc34465ae6be5dae2c","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e06dbdc34465ae6be5dae2c: start","job_status":"start","queue_duration_s":2.775204,"scheduling_latency_s":2.775217} gitlab # [ 138.752875] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e06dbdc34465ae6be5dae2c","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e06dbdc34465ae6be5dae2c: done: 0.009411 sec","job_status":"done","queue_duration_s":2.775204,"scheduling_latency_s":2.775217,"redis_calls":13,"redis_duration_s":0.001573,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009411,"completed_at":"2026-07-21T07:45:34.981Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.753991] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5814704cea658b518dd7c37f","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5814704cea658b518dd7c37f: start","job_status":"start","queue_duration_s":2.78559,"scheduling_latency_s":2.785603} gitlab # [ 138.760964] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5814704cea658b518dd7c37f","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5814704cea658b518dd7c37f: done: 0.009358 sec","job_status":"done","queue_duration_s":2.78559,"scheduling_latency_s":2.785603,"redis_calls":13,"redis_duration_s":0.001552,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001032,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009358,"completed_at":"2026-07-21T07:45:34.991Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.803413] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:34.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9b519f3d27468033188487b","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9b519f3d27468033188487b: start","job_status":"start","queue_duration_s":2.795895,"scheduling_latency_s":2.795909} gitlab # [ 138.841861] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9b519f3d27468033188487b","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9b519f3d27468033188487b: done: 0.009413 sec","job_status":"done","queue_duration_s":2.795895,"scheduling_latency_s":2.795909,"redis_calls":13,"redis_duration_s":0.00163,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009413,"completed_at":"2026-07-21T07:45:35.002Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.842893] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.003Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87adf5d2366e6d691f530fb1","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87adf5d2366e6d691f530fb1: start","job_status":"start","queue_duration_s":2.8063,"scheduling_latency_s":2.806315} gitlab # [ 138.850104] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87adf5d2366e6d691f530fb1","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87adf5d2366e6d691f530fb1: done: 0.009709 sec","job_status":"done","queue_duration_s":2.8063,"scheduling_latency_s":2.806315,"redis_calls":13,"redis_duration_s":0.001703,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000639,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009709,"completed_at":"2026-07-21T07:45:35.012Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.901744] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"442bf77269a7377721698905","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-442bf77269a7377721698905: start","job_status":"start","queue_duration_s":2.816857,"scheduling_latency_s":2.816877} gitlab # [ 138.902620] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"442bf77269a7377721698905","created_at":"2026-07-21T07:45:31.992Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-e4a01bf9052f4a03","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"20ba404bfa256392283af3089d2c4550","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.196Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-442bf77269a7377721698905: done: 0.009631 sec","job_status":"done","queue_duration_s":2.816857,"scheduling_latency_s":2.816877,"redis_calls":13,"redis_duration_s":0.0017339999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001183,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009631,"completed_at":"2026-07-21T07:45:35.023Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 138.966962] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.024Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"08b203c5806bb0f4e75496b0","created_at":"2026-07-21T07:45:32.193Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58f927c3190264398daacda3439bbd76","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.205Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-08b203c5806bb0f4e75496b0: start","job_status":"start","queue_duration_s":2.81873,"scheduling_latency_s":2.818743} gitlab # [ 138.967743] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.041Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"08b203c5806bb0f4e75496b0","created_at":"2026-07-21T07:45:32.193Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58f927c3190264398daacda3439bbd76","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.205Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-08b203c5806bb0f4e75496b0: done: 0.017111 sec","job_status":"done","queue_duration_s":2.81873,"scheduling_latency_s":2.818743,"redis_calls":12,"redis_duration_s":0.00163,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000306,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000553,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000300318,"exclusive_lock_hold_duration_s":0.004644446,"duration_s":0.017111,"completed_at":"2026-07-21T07:45:35.041Z","load_balancing_strategy":"primary","db_duration_s":0.002245,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.006757] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.042Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"0416aa32a2d0b569adcd4571","created_at":"2026-07-21T07:45:32.269Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b84f2369c818c289c76b58687a5c5b62","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.290Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-0416aa32a2d0b569adcd4571: start","job_status":"start","queue_duration_s":2.752098,"scheduling_latency_s":2.752111} gitlab # [ 139.007670] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.061Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"0416aa32a2d0b569adcd4571","created_at":"2026-07-21T07:45:32.269Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b84f2369c818c289c76b58687a5c5b62","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.290Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-0416aa32a2d0b569adcd4571: done: 0.019007 sec","job_status":"done","queue_duration_s":2.752098,"scheduling_latency_s":2.752111,"redis_calls":10,"redis_duration_s":0.001352,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000283,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000781,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000288,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0001936,"duration_s":0.019007,"completed_at":"2026-07-21T07:45:35.061Z","load_balancing_strategy":"primary","db_duration_s":0.003016,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.048721] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.062Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"4574f4be074b6295dd601ace","created_at":"2026-07-21T07:45:32.308Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ffd6ce649c1969971238d38b71b2814","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.323Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-4574f4be074b6295dd601ace: start","job_status":"start","queue_duration_s":2.738795,"scheduling_latency_s":2.738808} gitlab # [ 139.049531] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.186Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"4574f4be074b6295dd601ace","created_at":"2026-07-21T07:45:32.308Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6ffd6ce649c1969971238d38b71b2814","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.323Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-4574f4be074b6295dd601ace: done: 0.124419 sec","job_status":"done","queue_duration_s":2.738795,"scheduling_latency_s":2.738808,"redis_calls":12,"redis_duration_s":0.001973,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000291,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000812,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":123,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":80,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.101486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000309536,"exclusive_lock_hold_duration_s":0.110790897,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.124419,"completed_at":"2026-07-21T07:45:35.186Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026943,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.077628] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.187Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"9bd78d041fd81e1f534ec421","created_at":"2026-07-21T07:45:32.337Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9954592e66f8e9c04eb70797b11f2faf","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.354Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-9bd78d041fd81e1f534ec421: start","job_status":"start","queue_duration_s":2.833106,"scheduling_latency_s":2.833122} gitlab # [ 139.089147] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.265Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainVerificationCronWorker","jid":"9bd78d041fd81e1f534ec421","created_at":"2026-07-21T07:45:32.337Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9954592e66f8e9c04eb70797b11f2faf","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:4b3095972298b58b9c376f502255d63e0934f66e5921dfe9547414a5f347be92","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.354Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainVerificationCronWorker JID-9bd78d041fd81e1f534ec421: done: 0.077257 sec","job_status":"done","queue_duration_s":2.833106,"scheduling_latency_s":2.833122,"redis_calls":10,"redis_duration_s":0.0014,"redis_read_bytes":9,"redis_write_bytes":1097,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":309,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000846,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000294,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":218,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002112,"duration_s":0.077257,"completed_at":"2026-07-21T07:45:35.265Z","load_balancing_strategy":"primary","db_duration_s":0.060536,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.155080] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"20fef272c072daf41454c4e6","created_at":"2026-07-21T07:45:32.364Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd8d2484ea49098c002088a2f48bed0e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.379Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-20fef272c072daf41454c4e6: start","job_status":"start","queue_duration_s":2.886613,"scheduling_latency_s":2.886629} gitlab # [ 139.155826] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.281Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"20fef272c072daf41454c4e6","created_at":"2026-07-21T07:45:32.364Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd8d2484ea49098c002088a2f48bed0e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.379Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-20fef272c072daf41454c4e6: done: 0.015117 sec","job_status":"done","queue_duration_s":2.886613,"scheduling_latency_s":2.886629,"redis_calls":10,"redis_duration_s":0.001294,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000253,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000253,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000193321,"duration_s":0.015117,"completed_at":"2026-07-21T07:45:35.281Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001595,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.173283] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.282Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ed8f698b40feb9d4b92a014a","created_at":"2026-07-21T07:45:32.388Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b976f3c568e654df8257ada77ccb1000","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.424Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ed8f698b40feb9d4b92a014a: start","job_status":"start","queue_duration_s":2.857494,"scheduling_latency_s":2.857507} gitlab # [ 139.213091] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.296Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ed8f698b40feb9d4b92a014a","created_at":"2026-07-21T07:45:32.388Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b976f3c568e654df8257ada77ccb1000","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.424Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ed8f698b40feb9d4b92a014a: done: 0.014014 sec","job_status":"done","queue_duration_s":2.857494,"scheduling_latency_s":2.857507,"redis_calls":12,"redis_duration_s":0.001592,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000541,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000294,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000194159,"duration_s":0.014014,"completed_at":"2026-07-21T07:45:35.296Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000461,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.222538] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.297Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"99b16aaae78876af00030f5b","created_at":"2026-07-21T07:45:32.437Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3003432eb32e970186b1636606035551","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.456Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-99b16aaae78876af00030f5b: start","job_status":"start","queue_duration_s":2.840373,"scheduling_latency_s":2.840386} gitlab # [ 139.235626] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.304Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 139.238714] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.308Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"99b16aaae78876af00030f5b","created_at":"2026-07-21T07:45:32.437Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3003432eb32e970186b1636606035551","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.456Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-99b16aaae78876af00030f5b: done: 0.011426 sec","job_status":"done","queue_duration_s":2.840373,"scheduling_latency_s":2.840386,"redis_calls":10,"redis_duration_s":0.001282,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000257,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000202819,"duration_s":0.011426,"completed_at":"2026-07-21T07:45:35.308Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00023,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.282757] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.309Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ae25693566754f4a1f2c81ab","created_at":"2026-07-21T07:45:32.471Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"074dea518c5fc3bfa04ac9fb6c16d2b1","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.482Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ae25693566754f4a1f2c81ab: start","job_status":"start","queue_duration_s":2.827648,"scheduling_latency_s":2.827661} gitlab # [ 139.287913] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.340Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ae25693566754f4a1f2c81ab","created_at":"2026-07-21T07:45:32.471Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"074dea518c5fc3bfa04ac9fb6c16d2b1","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.482Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ae25693566754f4a1f2c81ab: done: 0.030988 sec","job_status":"done","queue_duration_s":2.827648,"scheduling_latency_s":2.827661,"redis_calls":12,"redis_duration_s":0.001641,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000245,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000655,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000305346,"exclusive_lock_hold_duration_s":0.0034428949999999996,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.030988,"completed_at":"2026-07-21T07:45:35.340Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001737,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.327931] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.488Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"890fb60536b2d51653fba18e","created_at":"2026-07-21T07:45:32.489Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42aaaf880d5baac97209834123a07779","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-21T07:45:32.500Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-890fb60536b2d51653fba18e: start","job_status":"start","queue_duration_s":2.987471,"scheduling_latency_s":2.987493} gitlab # [ 139.336996] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.521Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"890fb60536b2d51653fba18e","created_at":"2026-07-21T07:45:32.489Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"42aaaf880d5baac97209834123a07779","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-21T07:45:32.500Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-890fb60536b2d51653fba18e: done: 0.033343 sec","job_status":"done","queue_duration_s":2.987471,"scheduling_latency_s":2.987493,"redis_calls":10,"redis_duration_s":0.001425,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000278,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000226286,"duration_s":0.033343,"completed_at":"2026-07-21T07:45:35.521Z","load_balancing_strategy":"primary","db_duration_s":0.002337,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.397103] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.523Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"9e4d85a79286f348c9eef4b0","created_at":"2026-07-21T07:45:32.509Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87959793863afd0b29f4c922c1d0a86f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-21T07:45:32.515Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-9e4d85a79286f348c9eef4b0: start","job_status":"start","queue_duration_s":3.007488,"scheduling_latency_s":3.007502} gitlab # [ 139.397753] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.530Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 139.398268] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.535Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"9e4d85a79286f348c9eef4b0","created_at":"2026-07-21T07:45:32.509Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"87959793863afd0b29f4c922c1d0a86f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-21T07:45:32.515Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-9e4d85a79286f348c9eef4b0: done: 0.012817 sec","job_status":"done","queue_duration_s":3.007488,"scheduling_latency_s":3.007502,"redis_calls":10,"redis_duration_s":0.001913,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000262,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000774,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000877,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000229917,"duration_s":0.012817,"completed_at":"2026-07-21T07:45:35.535Z","load_balancing_strategy":"primary","db_duration_s":0.000242,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.451874] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.536Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"350da9e50536f5a7fd1a42ae","created_at":"2026-07-21T07:45:32.526Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"324c481afc7b6312d5e23963fcf825ad","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.532Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-350da9e50536f5a7fd1a42ae: start","job_status":"start","queue_duration_s":3.004227,"scheduling_latency_s":3.00424} gitlab # [ 139.452685] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.556Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"350da9e50536f5a7fd1a42ae","created_at":"2026-07-21T07:45:32.526Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"324c481afc7b6312d5e23963fcf825ad","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.532Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-350da9e50536f5a7fd1a42ae: done: 0.019519 sec","job_status":"done","queue_duration_s":3.004227,"scheduling_latency_s":3.00424,"redis_calls":16,"redis_duration_s":0.002105,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001075,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208965,"duration_s":0.019519,"completed_at":"2026-07-21T07:45:35.556Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003512,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.493260] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.557Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"753568d24e1dd60dde6848f7","created_at":"2026-07-21T07:45:32.537Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"452c5fbd59ae029b7515d17d0c9b6ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-21T07:45:32.542Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-753568d24e1dd60dde6848f7: start","job_status":"start","queue_duration_s":3.015693,"scheduling_latency_s":3.015707} gitlab # [ 139.493837] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.564Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 139.494518] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.568Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"753568d24e1dd60dde6848f7","created_at":"2026-07-21T07:45:32.537Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"452c5fbd59ae029b7515d17d0c9b6ddd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-21T07:45:32.542Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-753568d24e1dd60dde6848f7: done: 0.011292 sec","job_status":"done","queue_duration_s":3.015693,"scheduling_latency_s":3.015707,"redis_calls":10,"redis_duration_s":0.001287,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000276,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006332,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000195835,"duration_s":0.011292,"completed_at":"2026-07-21T07:45:35.568Z","load_balancing_strategy":"primary","db_duration_s":0.000221,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.533594] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.569Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"273f7a367177deabe083709b","created_at":"2026-07-21T07:45:32.546Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dcb4cc9548f547ecf42ca110e0d54ea8","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.551Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-273f7a367177deabe083709b: start","job_status":"start","queue_duration_s":3.018313,"scheduling_latency_s":3.018326} gitlab # [ 139.534211] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.582Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"273f7a367177deabe083709b","created_at":"2026-07-21T07:45:32.546Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dcb4cc9548f547ecf42ca110e0d54ea8","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.551Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-273f7a367177deabe083709b: done: 0.013029 sec","job_status":"done","queue_duration_s":3.018313,"scheduling_latency_s":3.018326,"redis_calls":11,"redis_duration_s":0.001524,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00081,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000181,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000252,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000217346,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.013029,"completed_at":"2026-07-21T07:45:35.582Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000226,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.588887] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.583Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d04c9cada7c66f1d9f83bfe7","created_at":"2026-07-21T07:45:32.556Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d66fa4b760826c4704c218d35aa4458d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.560Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d04c9cada7c66f1d9f83bfe7: start","job_status":"start","queue_duration_s":3.022832,"scheduling_latency_s":3.022852} gitlab # [ 139.589646] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.595Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d04c9cada7c66f1d9f83bfe7","created_at":"2026-07-21T07:45:32.556Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d66fa4b760826c4704c218d35aa4458d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.560Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d04c9cada7c66f1d9f83bfe7: done: 0.011922 sec","job_status":"done","queue_duration_s":3.022832,"scheduling_latency_s":3.022852,"redis_calls":10,"redis_duration_s":0.0013679999999999999,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000276,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000785,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000307,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000196673,"duration_s":0.011922,"completed_at":"2026-07-21T07:45:35.595Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000242,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.632921] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.596Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0abce992daea1d853989d256","created_at":"2026-07-21T07:45:32.568Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7cb6a94ba24881313ed4b60c39c2fb31","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.573Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0abce992daea1d853989d256: start","job_status":"start","queue_duration_s":3.023365,"scheduling_latency_s":3.023378} gitlab # [ 139.633663] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.602Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 139.634804] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0abce992daea1d853989d256","created_at":"2026-07-21T07:45:32.568Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7cb6a94ba24881313ed4b60c39c2fb31","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.573Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0abce992daea1d853989d256: done: 0.010403 sec","job_status":"done","queue_duration_s":3.023365,"scheduling_latency_s":3.023378,"redis_calls":10,"redis_duration_s":0.001292,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000274,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000761,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000257,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000191924,"duration_s":0.010403,"completed_at":"2026-07-21T07:45:35.606Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000229,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.671924] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.607Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"a7b063e3a2f3609f6d5716e4","created_at":"2026-07-21T07:45:32.577Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e775a22df9c1e6021e2b475f4d6d8781","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.582Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-a7b063e3a2f3609f6d5716e4: start","job_status":"start","queue_duration_s":3.025717,"scheduling_latency_s":3.02573} gitlab # [ 139.711651] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.623Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"StuckMergeJobsWorker","jid":"a7b063e3a2f3609f6d5716e4","created_at":"2026-07-21T07:45:32.577Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e775a22df9c1e6021e2b475f4d6d8781","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ce304458c2d63d5d84855b65f51692704db52bf763a718b10c08a2bd5f588f06","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.582Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"StuckMergeJobsWorker JID-a7b063e3a2f3609f6d5716e4: done: 0.016092 sec","job_status":"done","queue_duration_s":3.025717,"scheduling_latency_s":3.02573,"redis_calls":11,"redis_duration_s":0.001494,"redis_read_bytes":10,"redis_write_bytes":1022,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000827,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000421,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":225,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000225168,"duration_s":0.016092,"completed_at":"2026-07-21T07:45:35.623Z","load_balancing_strategy":"primary","db_duration_s":0.002051,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.718052] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.624Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"7b691d1aa15091f42310203a","created_at":"2026-07-21T07:45:32.590Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4052c6f1a7eb298779c10a7e13843ecb","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.594Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-7b691d1aa15091f42310203a: start","job_status":"start","queue_duration_s":3.030225,"scheduling_latency_s":3.030238} gitlab # [ 139.730578] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.643Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"7b691d1aa15091f42310203a","created_at":"2026-07-21T07:45:32.590Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4052c6f1a7eb298779c10a7e13843ecb","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-21T07:45:32.594Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-7b691d1aa15091f42310203a: done: 0.019149 sec","job_status":"done","queue_duration_s":3.030225,"scheduling_latency_s":3.030238,"redis_calls":10,"redis_duration_s":0.0013670000000000002,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000255,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000814,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000298,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000207289,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.019149,"completed_at":"2026-07-21T07:45:35.643Z","load_balancing_strategy":"primary","db_duration_s":0.00711,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.775772] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.644Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6d394a3acb3f868e3d6fda9b","created_at":"2026-07-21T07:45:33.104Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-ec44be7241ef451d","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9074c800357ef8c8cd18dae0cf592ca7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:33.113Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6d394a3acb3f868e3d6fda9b: start","job_status":"start","queue_duration_s":2.531678,"scheduling_latency_s":2.531691} gitlab # [ 139.778033] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.654Z","message":"No existing merge request to be cleaned up."} gitlab # [ 139.779635] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.663Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"6d394a3acb3f868e3d6fda9b","created_at":"2026-07-21T07:45:33.104Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-ec44be7241ef451d","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9074c800357ef8c8cd18dae0cf592ca7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:33.113Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-6d394a3acb3f868e3d6fda9b: done: 0.018566 sec","job_status":"done","queue_duration_s":2.531678,"scheduling_latency_s":2.531691,"redis_calls":13,"redis_duration_s":0.001778,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000246,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000621,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000911,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000213435,"duration_s":0.018566,"completed_at":"2026-07-21T07:45:35.663Z","load_balancing_strategy":"primary","db_duration_s":0.002467,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.837258] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.664Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b190c091a0c3924a36fdea1a","created_at":"2026-07-21T07:45:33.104Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-ec44be7241ef451d","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9074c800357ef8c8cd18dae0cf592ca7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:33.113Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b190c091a0c3924a36fdea1a: start","job_status":"start","queue_duration_s":2.551129,"scheduling_latency_s":2.551142} gitlab # [ 139.837845] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.670Z","message":"No existing merge request to be cleaned up."} gitlab # [ 139.838522] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.675Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b190c091a0c3924a36fdea1a","created_at":"2026-07-21T07:45:33.104Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-ec44be7241ef451d","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9074c800357ef8c8cd18dae0cf592ca7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:33.113Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b190c091a0c3924a36fdea1a: done: 0.010926 sec","job_status":"done","queue_duration_s":2.551129,"scheduling_latency_s":2.551142,"redis_calls":10,"redis_duration_s":0.001413,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000834,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010926,"completed_at":"2026-07-21T07:45:35.675Z","load_balancing_strategy":"primary","db_duration_s":0.000949,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.860601] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.675Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"48e513de34ab4f86f3280d06","created_at":"2026-07-21T07:45:33.104Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-ec44be7241ef451d","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9074c800357ef8c8cd18dae0cf592ca7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:33.113Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-48e513de34ab4f86f3280d06: start","job_status":"start","queue_duration_s":2.562883,"scheduling_latency_s":2.562897} gitlab # [ 139.871451] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.682Z","message":"No existing merge request to be cleaned up."} gitlab # [ 139.872732] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.686Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"48e513de34ab4f86f3280d06","created_at":"2026-07-21T07:45:33.104Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-ec44be7241ef451d","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9074c800357ef8c8cd18dae0cf592ca7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:33.113Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-48e513de34ab4f86f3280d06: done: 0.010435 sec","job_status":"done","queue_duration_s":2.562883,"scheduling_latency_s":2.562897,"redis_calls":10,"redis_duration_s":0.001417,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000884,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006869,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010435,"completed_at":"2026-07-21T07:45:35.686Z","load_balancing_strategy":"primary","db_duration_s":0.000959,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 139.921710] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.687Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f0aa397148234f8448993fdd","created_at":"2026-07-21T07:45:33.104Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-ec44be7241ef451d","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9074c800357ef8c8cd18dae0cf592ca7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:33.113Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f0aa397148234f8448993fdd: start","job_status":"start","queue_duration_s":2.574135,"scheduling_latency_s":2.574148} gitlab # [ 139.922549] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.693Z","message":"No existing merge request to be cleaned up."} gitlab # [ 139.923046] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:45:35.697Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f0aa397148234f8448993fdd","created_at":"2026-07-21T07:45:33.104Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-ec44be7241ef451d","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9074c800357ef8c8cd18dae0cf592ca7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:45:33.113Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f0aa397148234f8448993fdd: done: 0.010175 sec","job_status":"done","queue_duration_s":2.574135,"scheduling_latency_s":2.574148,"redis_calls":10,"redis_duration_s":0.0012959999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000785,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010175,"completed_at":"2026-07-21T07:45:35.697Z","load_balancing_strategy":"primary","db_duration_s":0.000937,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 154.405821] setup-gitlab-start[2609]: W, [2026-07-21T07:45:51.096295 #2609] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 164.564148] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:01.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b2eae582a56b060858f9d7a8","created_at":"2026-07-21T07:46:01.202Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.251Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b2eae582a56b060858f9d7a8: start","job_status":"start","queue_duration_s":0.002443,"scheduling_latency_s":0.002491} gitlab # [ 164.821499] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:01.512Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b2eae582a56b060858f9d7a8","created_at":"2026-07-21T07:46:01.202Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.251Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b2eae582a56b060858f9d7a8: done: 0.25796 sec","job_status":"done","queue_duration_s":0.002443,"scheduling_latency_s":0.002491,"redis_calls":60,"redis_duration_s":0.08582899999999999,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.003179,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.079951,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002699,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.098297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.25796,"completed_at":"2026-07-21T07:46:01.511Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 164.890218] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:01.581Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"2d890afedd8229a3a7608821","created_at":"2026-07-21T07:46:01.261Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"466b1ad072ab7eee7bb175d99df7a279","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.271Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-2d890afedd8229a3a7608821: start","job_status":"start","queue_duration_s":0.309895,"scheduling_latency_s":0.309941} gitlab # [ 164.947553] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:01.638Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"2d890afedd8229a3a7608821","created_at":"2026-07-21T07:46:01.261Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"466b1ad072ab7eee7bb175d99df7a279","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.271Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-2d890afedd8229a3a7608821: done: 0.05721 sec","job_status":"done","queue_duration_s":0.309895,"scheduling_latency_s":0.309941,"redis_calls":10,"redis_duration_s":0.007634999999999999,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005212,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002423,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018556,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.022504536,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.05721,"completed_at":"2026-07-21T07:46:01.638Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031487,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 164.996760] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:01.687Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5bf75353fcfbbff1366d6731","created_at":"2026-07-21T07:46:01.283Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3134def553114af0389dff862cd821b0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.301Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5bf75353fcfbbff1366d6731: start","job_status":"start","queue_duration_s":0.386536,"scheduling_latency_s":0.386579} gitlab # [ 165.118925] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:01.809Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"5bf75353fcfbbff1366d6731","created_at":"2026-07-21T07:46:01.283Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3134def553114af0389dff862cd821b0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.301Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-5bf75353fcfbbff1366d6731: done: 0.121849 sec","job_status":"done","queue_duration_s":0.386536,"scheduling_latency_s":0.386579,"redis_calls":12,"redis_duration_s":0.009925,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000929,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004687,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004309,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001042031,"exclusive_lock_hold_duration_s":0.021844955,"duration_s":0.121849,"completed_at":"2026-07-21T07:46:01.809Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020855,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.195232] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:01.886Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"de0fc09cd8aa5b6acd9aa766","created_at":"2026-07-21T07:46:01.321Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"231e5555255a4de22bab248998557a45","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.325Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-de0fc09cd8aa5b6acd9aa766: start","job_status":"start","queue_duration_s":0.56014,"scheduling_latency_s":0.560183} gitlab # [ 165.501121] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.191Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"de0fc09cd8aa5b6acd9aa766","created_at":"2026-07-21T07:46:01.321Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"231e5555255a4de22bab248998557a45","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.325Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-de0fc09cd8aa5b6acd9aa766: done: 0.305818 sec","job_status":"done","queue_duration_s":0.56014,"scheduling_latency_s":0.560183,"redis_calls":571,"redis_duration_s":0.11810699999999999,"redis_read_bytes":569,"redis_write_bytes":49487,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001353,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":566,"redis_shared_state_duration_s":0.116754,"redis_shared_state_read_bytes":565,"redis_shared_state_write_bytes":49017,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.217496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.305818,"completed_at":"2026-07-21T07:46:02.191Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.580562] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.270Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"48a07fce407f06d21fe090fd","created_at":"2026-07-21T07:46:01.335Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c04d941ecbcb5239fd521579d83be77","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.347Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-48a07fce407f06d21fe090fd: start","job_status":"start","queue_duration_s":0.923841,"scheduling_latency_s":0.923871} gitlab # [ 165.595438] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.286Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"48a07fce407f06d21fe090fd","created_at":"2026-07-21T07:46:01.335Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c04d941ecbcb5239fd521579d83be77","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.347Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-48a07fce407f06d21fe090fd: done: 0.015285 sec","job_status":"done","queue_duration_s":0.923841,"scheduling_latency_s":0.923871,"redis_calls":12,"redis_duration_s":0.00245,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000455,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00041,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001198,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000387,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000282158,"duration_s":0.015285,"completed_at":"2026-07-21T07:46:02.286Z","load_balancing_strategy":"primary","db_duration_s":0.000657,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.634597] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.325Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"539b644a8a3e190abf5f9a8a","created_at":"2026-07-21T07:46:01.363Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e60f962012e7cdfdbef591f3a053b8a7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.366Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-539b644a8a3e190abf5f9a8a: start","job_status":"start","queue_duration_s":0.958296,"scheduling_latency_s":0.958342} gitlab # [ 165.703196] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.393Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"539b644a8a3e190abf5f9a8a","created_at":"2026-07-21T07:46:01.363Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e60f962012e7cdfdbef591f3a053b8a7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.366Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-539b644a8a3e190abf5f9a8a: done: 0.068534 sec","job_status":"done","queue_duration_s":0.958296,"scheduling_latency_s":0.958342,"redis_calls":9,"redis_duration_s":0.003225,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001691,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001534,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00082748,"exclusive_lock_hold_duration_s":0.045865606,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.068534,"completed_at":"2026-07-21T07:46:02.393Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.036019,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.730138] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.421Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"4d629966f8b3c17c31d5aa64","created_at":"2026-07-21T07:46:01.376Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"817362336569c4ccedc93068ebef469d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.380Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-4d629966f8b3c17c31d5aa64: start","job_status":"start","queue_duration_s":1.040895,"scheduling_latency_s":1.040932} gitlab # [ 165.765241] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.455Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"4d629966f8b3c17c31d5aa64","created_at":"2026-07-21T07:46:01.376Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"817362336569c4ccedc93068ebef469d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.380Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-4d629966f8b3c17c31d5aa64: done: 0.034989 sec","job_status":"done","queue_duration_s":1.040895,"scheduling_latency_s":1.040932,"redis_calls":14,"redis_duration_s":0.004111,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000387,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002816,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000908,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034989,"completed_at":"2026-07-21T07:46:02.455Z","load_balancing_strategy":"primary","db_duration_s":0.001602,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.769524] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.457Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"6a2dda81848045ff2e021fc1","created_at":"2026-07-21T07:46:01.389Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"583bd0e91eadb80b3b89666958e95f04","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.393Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-6a2dda81848045ff2e021fc1: start","job_status":"start","queue_duration_s":1.06424,"scheduling_latency_s":1.064265} gitlab # [ 165.783945] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.465Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 165.787537] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.468Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"6a2dda81848045ff2e021fc1","created_at":"2026-07-21T07:46:01.389Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"583bd0e91eadb80b3b89666958e95f04","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.393Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-6a2dda81848045ff2e021fc1: done: 0.011284 sec","job_status":"done","queue_duration_s":1.06424,"scheduling_latency_s":1.064265,"redis_calls":7,"redis_duration_s":0.001374,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00102,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000354,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011284,"completed_at":"2026-07-21T07:46:02.468Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.845838] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.469Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"dd241990d7e2acd815c2c301","created_at":"2026-07-21T07:46:01.409Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3432ade3d560d41153d38bed12ac87c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.413Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-dd241990d7e2acd815c2c301: start","job_status":"start","queue_duration_s":1.0558,"scheduling_latency_s":1.055821} gitlab # [ 165.846829] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.474Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 165.848247] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"dd241990d7e2acd815c2c301","created_at":"2026-07-21T07:46:01.409Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3432ade3d560d41153d38bed12ac87c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.413Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-dd241990d7e2acd815c2c301: done: 0.007353 sec","job_status":"done","queue_duration_s":1.0558,"scheduling_latency_s":1.055821,"redis_calls":7,"redis_duration_s":0.000937,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000674,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007353,"completed_at":"2026-07-21T07:46:02.477Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.912299] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"174144dd0d9806ff6aba9683","created_at":"2026-07-21T07:46:01.419Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3ee812313c1b90a550f5bbd2421c1d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.422Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-174144dd0d9806ff6aba9683: start","job_status":"start","queue_duration_s":1.055823,"scheduling_latency_s":1.055835} gitlab # [ 165.913121] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.506Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"174144dd0d9806ff6aba9683","created_at":"2026-07-21T07:46:01.419Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d3ee812313c1b90a550f5bbd2421c1d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.422Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-174144dd0d9806ff6aba9683: done: 0.028959 sec","job_status":"done","queue_duration_s":1.055823,"scheduling_latency_s":1.055835,"redis_calls":9,"redis_duration_s":0.001401,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000678,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000723,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00034781,"exclusive_lock_hold_duration_s":0.010993017,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.028959,"completed_at":"2026-07-21T07:46:02.506Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01568,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 165.931554] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.507Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"cf44c2699bfcbb0c9c45214b","created_at":"2026-07-21T07:46:01.441Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2bde3321391109a7eeb78f5cb62f5aee","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.443Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-cf44c2699bfcbb0c9c45214b: start","job_status":"start","queue_duration_s":1.064019,"scheduling_latency_s":1.064032} gitlab # [ 165.943964] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.518Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"cf44c2699bfcbb0c9c45214b","created_at":"2026-07-21T07:46:01.441Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2bde3321391109a7eeb78f5cb62f5aee","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.443Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-cf44c2699bfcbb0c9c45214b: done: 0.010217 sec","job_status":"done","queue_duration_s":1.064019,"scheduling_latency_s":1.064032,"redis_calls":9,"redis_duration_s":0.0016159999999999998,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000766,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000352838,"exclusive_lock_hold_duration_s":0.001847163,"duration_s":0.010217,"completed_at":"2026-07-21T07:46:02.518Z","load_balancing_strategy":"primary","db_duration_s":0.000826,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.011600] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.519Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ddc9951bf3896238cf503498","created_at":"2026-07-21T07:46:01.462Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0412fced930770ad797a2aea5912945","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.466Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ddc9951bf3896238cf503498: start","job_status":"start","queue_duration_s":1.053065,"scheduling_latency_s":1.053077} gitlab # [ 166.012509] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.676Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ddc9951bf3896238cf503498","created_at":"2026-07-21T07:46:01.462Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0412fced930770ad797a2aea5912945","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.466Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ddc9951bf3896238cf503498: done: 0.157016 sec","job_status":"done","queue_duration_s":1.053065,"scheduling_latency_s":1.053077,"redis_calls":11,"redis_duration_s":0.0024029999999999998,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000712,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000801,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":124,"db_write_count":0,"db_cached_count":80,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":124,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":80,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":124,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":80,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.097433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000355632,"exclusive_lock_hold_duration_s":0.145232984,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.157016,"completed_at":"2026-07-21T07:46:02.676Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.06487,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.052633] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.677Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::PipelineArtifacts::ExpireArtifactsWorker","jid":"ffc611c0484b05a3feaff016","created_at":"2026-07-21T07:46:01.472Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"88fe0d5d4b5823e87d033f6627bdf23e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:35dc78c13bdf78a98c09fc2795030a4961be87ad882288b77a6a3c9be677e4d5","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.478Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::ExpireArtifactsWorker JID-ffc611c0484b05a3feaff016: start","job_status":"start","queue_duration_s":1.198617,"scheduling_latency_s":1.198631} gitlab # [ 166.053324] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.694Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::PipelineArtifacts::ExpireArtifactsWorker","jid":"ffc611c0484b05a3feaff016","created_at":"2026-07-21T07:46:01.472Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"88fe0d5d4b5823e87d033f6627bdf23e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:35dc78c13bdf78a98c09fc2795030a4961be87ad882288b77a6a3c9be677e4d5","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.478Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::ExpireArtifactsWorker JID-ffc611c0484b05a3feaff016: done: 0.017311 sec","job_status":"done","queue_duration_s":1.198617,"scheduling_latency_s":1.198631,"redis_calls":12,"redis_duration_s":0.001457,"redis_read_bytes":11,"redis_write_bytes":1481,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000678,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000537,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":548,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000269588,"exclusive_lock_hold_duration_s":0.00613793,"extra.ci_pipeline_artifacts_expire_artifacts_worker.destroyed_pipeline_artifacts_count":0,"duration_s":0.017311,"completed_at":"2026-07-21T07:46:02.694Z","load_balancing_strategy":"primary","db_duration_s":0.003089,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.094745] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.695Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"405c44c9a1c93992ac8cd861","created_at":"2026-07-21T07:46:01.487Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f3cf4a5016b23b79d40b52e17af474e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.489Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-405c44c9a1c93992ac8cd861: start","job_status":"start","queue_duration_s":1.205683,"scheduling_latency_s":1.205696} gitlab # [ 166.096952] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.703Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"405c44c9a1c93992ac8cd861","created_at":"2026-07-21T07:46:01.487Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f3cf4a5016b23b79d40b52e17af474e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.489Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-405c44c9a1c93992ac8cd861: done: 0.008171 sec","job_status":"done","queue_duration_s":1.205683,"scheduling_latency_s":1.205696,"redis_calls":7,"redis_duration_s":0.000932,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000227,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008171,"completed_at":"2026-07-21T07:46:02.703Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000607,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.150901] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b0c9862ad79ab2bf58b2c2f","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b0c9862ad79ab2bf58b2c2f: start","job_status":"start","queue_duration_s":1.211523,"scheduling_latency_s":1.211536} gitlab # [ 166.151687] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b0c9862ad79ab2bf58b2c2f","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b0c9862ad79ab2bf58b2c2f: done: 0.009103 sec","job_status":"done","queue_duration_s":1.211523,"scheduling_latency_s":1.211536,"redis_calls":13,"redis_duration_s":0.00146,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00046,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009103,"completed_at":"2026-07-21T07:46:02.714Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.180672] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48a704404629d0aa73bbf554","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48a704404629d0aa73bbf554: start","job_status":"start","queue_duration_s":1.221407,"scheduling_latency_s":1.22142} gitlab # [ 166.225782] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48a704404629d0aa73bbf554","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48a704404629d0aa73bbf554: done: 0.008728 sec","job_status":"done","queue_duration_s":1.221407,"scheduling_latency_s":1.22142,"redis_calls":13,"redis_duration_s":0.001391,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000462,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008728,"completed_at":"2026-07-21T07:46:02.723Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.226919] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cafea8deffc5a3d2f7b3f27","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cafea8deffc5a3d2f7b3f27: start","job_status":"start","queue_duration_s":1.230882,"scheduling_latency_s":1.230894} gitlab # [ 166.266405] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.733Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cafea8deffc5a3d2f7b3f27","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cafea8deffc5a3d2f7b3f27: done: 0.008769 sec","job_status":"done","queue_duration_s":1.230882,"scheduling_latency_s":1.230894,"redis_calls":13,"redis_duration_s":0.001418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000449,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008769,"completed_at":"2026-07-21T07:46:02.733Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.306446] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.734Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9163ae3ebc2b0e822561a271","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9163ae3ebc2b0e822561a271: start","job_status":"start","queue_duration_s":1.241449,"scheduling_latency_s":1.241462} gitlab # [ 166.307478] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9163ae3ebc2b0e822561a271","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9163ae3ebc2b0e822561a271: done: 0.008697 sec","job_status":"done","queue_duration_s":1.241449,"scheduling_latency_s":1.241462,"redis_calls":13,"redis_duration_s":0.001393,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000471,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005896,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008697,"completed_at":"2026-07-21T07:46:02.743Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.355266] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"193df6a70d2a3ec877f35fdb","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-193df6a70d2a3ec877f35fdb: start","job_status":"start","queue_duration_s":1.250976,"scheduling_latency_s":1.250988} gitlab # [ 166.356258] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"193df6a70d2a3ec877f35fdb","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-193df6a70d2a3ec877f35fdb: done: 0.009682 sec","job_status":"done","queue_duration_s":1.250976,"scheduling_latency_s":1.250988,"redis_calls":13,"redis_duration_s":0.002219,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000658,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001561,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009682,"completed_at":"2026-07-21T07:46:02.754Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.369093] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32104d5e41eefdc94160ee85","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32104d5e41eefdc94160ee85: start","job_status":"start","queue_duration_s":1.261551,"scheduling_latency_s":1.261563} gitlab # [ 166.411417] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.764Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32104d5e41eefdc94160ee85","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32104d5e41eefdc94160ee85: done: 0.009406 sec","job_status":"done","queue_duration_s":1.261551,"scheduling_latency_s":1.261563,"redis_calls":13,"redis_duration_s":0.0020610000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001417,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009406,"completed_at":"2026-07-21T07:46:02.764Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.414724] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15e23c6a6e776107ce98d74f","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15e23c6a6e776107ce98d74f: start","job_status":"start","queue_duration_s":1.271889,"scheduling_latency_s":1.271902} gitlab # [ 166.426377] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"15e23c6a6e776107ce98d74f","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-15e23c6a6e776107ce98d74f: done: 0.009864 sec","job_status":"done","queue_duration_s":1.271889,"scheduling_latency_s":1.271902,"redis_calls":13,"redis_duration_s":0.002105,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000691,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001414,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009864,"completed_at":"2026-07-21T07:46:02.775Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.484155] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d25f75bdb684aa38eff49e7b","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d25f75bdb684aa38eff49e7b: start","job_status":"start","queue_duration_s":1.282679,"scheduling_latency_s":1.282691} gitlab # [ 166.484850] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d25f75bdb684aa38eff49e7b","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d25f75bdb684aa38eff49e7b: done: 0.009886 sec","job_status":"done","queue_duration_s":1.282679,"scheduling_latency_s":1.282691,"redis_calls":13,"redis_duration_s":0.0024000000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000729,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001671,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009886,"completed_at":"2026-07-21T07:46:02.785Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.508207] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0242c0731de65a219bc9ceba","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0242c0731de65a219bc9ceba: start","job_status":"start","queue_duration_s":1.293374,"scheduling_latency_s":1.293386} gitlab # [ 166.519676] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0242c0731de65a219bc9ceba","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0242c0731de65a219bc9ceba: done: 0.008783 sec","job_status":"done","queue_duration_s":1.293374,"scheduling_latency_s":1.293386,"redis_calls":13,"redis_duration_s":0.001625,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008783,"completed_at":"2026-07-21T07:46:02.795Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.587651] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f357fa2a26de39f2bac40928","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f357fa2a26de39f2bac40928: start","job_status":"start","queue_duration_s":1.302966,"scheduling_latency_s":1.302978} gitlab # [ 166.588550] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f357fa2a26de39f2bac40928","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f357fa2a26de39f2bac40928: done: 0.010891 sec","job_status":"done","queue_duration_s":1.302966,"scheduling_latency_s":1.302978,"redis_calls":13,"redis_duration_s":0.0017239999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000674,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00689,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010891,"completed_at":"2026-07-21T07:46:02.807Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.627169] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.808Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6bad75a16a8a6477f55ee99","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6bad75a16a8a6477f55ee99: start","job_status":"start","queue_duration_s":1.315062,"scheduling_latency_s":1.315081} gitlab # [ 166.627780] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6bad75a16a8a6477f55ee99","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6bad75a16a8a6477f55ee99: done: 0.011198 sec","job_status":"done","queue_duration_s":1.315062,"scheduling_latency_s":1.315081,"redis_calls":13,"redis_duration_s":0.001474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011198,"completed_at":"2026-07-21T07:46:02.819Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.666794] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47d73ba0fc8671ccf264c1f5","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47d73ba0fc8671ccf264c1f5: start","job_status":"start","queue_duration_s":1.327059,"scheduling_latency_s":1.327073} gitlab # [ 166.667717] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.830Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47d73ba0fc8671ccf264c1f5","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47d73ba0fc8671ccf264c1f5: done: 0.009829 sec","job_status":"done","queue_duration_s":1.327059,"scheduling_latency_s":1.327073,"redis_calls":13,"redis_duration_s":0.001804,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000643,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009829,"completed_at":"2026-07-21T07:46:02.830Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.692808] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fa3b92a6696a62c09d304d2","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fa3b92a6696a62c09d304d2: start","job_status":"start","queue_duration_s":1.337784,"scheduling_latency_s":1.337798} gitlab # [ 166.704498] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.840Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fa3b92a6696a62c09d304d2","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fa3b92a6696a62c09d304d2: done: 0.008951 sec","job_status":"done","queue_duration_s":1.337784,"scheduling_latency_s":1.337798,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00046,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008951,"completed_at":"2026-07-21T07:46:02.840Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.752706] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.841Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e6f69167fccf3aeba1e4aac","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e6f69167fccf3aeba1e4aac: start","job_status":"start","queue_duration_s":1.347668,"scheduling_latency_s":1.34768} gitlab # [ 166.753528] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e6f69167fccf3aeba1e4aac","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e6f69167fccf3aeba1e4aac: done: 0.008863 sec","job_status":"done","queue_duration_s":1.347668,"scheduling_latency_s":1.34768,"redis_calls":13,"redis_duration_s":0.001562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008863,"completed_at":"2026-07-21T07:46:02.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.803691] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5336287eea2ed12cce48f6f","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5336287eea2ed12cce48f6f: start","job_status":"start","queue_duration_s":1.357345,"scheduling_latency_s":1.357358} gitlab # [ 166.804453] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5336287eea2ed12cce48f6f","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5336287eea2ed12cce48f6f: done: 0.008847 sec","job_status":"done","queue_duration_s":1.357345,"scheduling_latency_s":1.357358,"redis_calls":13,"redis_duration_s":0.0015209999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005412,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008847,"completed_at":"2026-07-21T07:46:02.859Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.851028] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.860Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fb5be4d8fd8aab1cf516847","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fb5be4d8fd8aab1cf516847: start","job_status":"start","queue_duration_s":1.366994,"scheduling_latency_s":1.367006} gitlab # [ 166.851848] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fb5be4d8fd8aab1cf516847","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fb5be4d8fd8aab1cf516847: done: 0.00906 sec","job_status":"done","queue_duration_s":1.366994,"scheduling_latency_s":1.367006,"redis_calls":13,"redis_duration_s":0.001523,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005656,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00906,"completed_at":"2026-07-21T07:46:02.869Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.902792] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.870Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cda96b23da2b0c10d10bd4e","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cda96b23da2b0c10d10bd4e: start","job_status":"start","queue_duration_s":1.376817,"scheduling_latency_s":1.37683} gitlab # [ 166.903761] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.879Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cda96b23da2b0c10d10bd4e","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cda96b23da2b0c10d10bd4e: done: 0.009021 sec","job_status":"done","queue_duration_s":1.376817,"scheduling_latency_s":1.37683,"redis_calls":13,"redis_duration_s":0.001484,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009021,"completed_at":"2026-07-21T07:46:02.879Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 166.957286] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53b2fdead3090ffebc14a4ec","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53b2fdead3090ffebc14a4ec: start","job_status":"start","queue_duration_s":1.386644,"scheduling_latency_s":1.386656} gitlab # [ 166.957901] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53b2fdead3090ffebc14a4ec","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53b2fdead3090ffebc14a4ec: done: 0.008649 sec","job_status":"done","queue_duration_s":1.386644,"scheduling_latency_s":1.386656,"redis_calls":13,"redis_duration_s":0.0015379999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005478,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008649,"completed_at":"2026-07-21T07:46:02.888Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.006720] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc48ea1bcb7d3504f3fa6ef6","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc48ea1bcb7d3504f3fa6ef6: start","job_status":"start","queue_duration_s":1.396108,"scheduling_latency_s":1.39612} gitlab # [ 167.007633] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc48ea1bcb7d3504f3fa6ef6","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc48ea1bcb7d3504f3fa6ef6: done: 0.008728 sec","job_status":"done","queue_duration_s":1.396108,"scheduling_latency_s":1.39612,"redis_calls":13,"redis_duration_s":0.001477,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005539,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008728,"completed_at":"2026-07-21T07:46:02.898Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.048230] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7067b523c25ac88cfc11b88","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7067b523c25ac88cfc11b88: start","job_status":"start","queue_duration_s":1.405672,"scheduling_latency_s":1.405685} gitlab # [ 167.048826] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7067b523c25ac88cfc11b88","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7067b523c25ac88cfc11b88: done: 0.008747 sec","job_status":"done","queue_duration_s":1.405672,"scheduling_latency_s":1.405685,"redis_calls":13,"redis_duration_s":0.001556,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008747,"completed_at":"2026-07-21T07:46:02.907Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.062653] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e08f9610e6807786a9c9ef2e","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e08f9610e6807786a9c9ef2e: start","job_status":"start","queue_duration_s":1.415213,"scheduling_latency_s":1.415226} gitlab # [ 167.107759] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e08f9610e6807786a9c9ef2e","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e08f9610e6807786a9c9ef2e: done: 0.00872 sec","job_status":"done","queue_duration_s":1.415213,"scheduling_latency_s":1.415226,"redis_calls":13,"redis_duration_s":0.0015049999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000488,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00872,"completed_at":"2026-07-21T07:46:02.917Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.108698] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28afc99c471fbd42a6a7ec2b","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28afc99c471fbd42a6a7ec2b: start","job_status":"start","queue_duration_s":1.424727,"scheduling_latency_s":1.42474} gitlab # [ 167.150664] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.927Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28afc99c471fbd42a6a7ec2b","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28afc99c471fbd42a6a7ec2b: done: 0.009066 sec","job_status":"done","queue_duration_s":1.424727,"scheduling_latency_s":1.42474,"redis_calls":13,"redis_duration_s":0.001591,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009066,"completed_at":"2026-07-21T07:46:02.927Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.151811] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.928Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a9f58132ed398bd44ccb9b8","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a9f58132ed398bd44ccb9b8: start","job_status":"start","queue_duration_s":1.4346,"scheduling_latency_s":1.434613} gitlab # [ 167.162744] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a9f58132ed398bd44ccb9b8","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a9f58132ed398bd44ccb9b8: done: 0.008886 sec","job_status":"done","queue_duration_s":1.4346,"scheduling_latency_s":1.434613,"redis_calls":13,"redis_duration_s":0.001608,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008886,"completed_at":"2026-07-21T07:46:02.936Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.216217] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80ce98f1bf0ea8e9e2e0ae6c","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80ce98f1bf0ea8e9e2e0ae6c: start","job_status":"start","queue_duration_s":1.444285,"scheduling_latency_s":1.444306} gitlab # [ 167.216793] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80ce98f1bf0ea8e9e2e0ae6c","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80ce98f1bf0ea8e9e2e0ae6c: done: 0.008857 sec","job_status":"done","queue_duration_s":1.444285,"scheduling_latency_s":1.444306,"redis_calls":13,"redis_duration_s":0.001615,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008857,"completed_at":"2026-07-21T07:46:02.946Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.257562] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5422fcf2cd7ef8796184025","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5422fcf2cd7ef8796184025: start","job_status":"start","queue_duration_s":1.454478,"scheduling_latency_s":1.454491} gitlab # [ 167.258151] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b5422fcf2cd7ef8796184025","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b5422fcf2cd7ef8796184025: done: 0.009872 sec","job_status":"done","queue_duration_s":1.454478,"scheduling_latency_s":1.454491,"redis_calls":13,"redis_duration_s":0.002637,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000962,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001675,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009872,"completed_at":"2026-07-21T07:46:02.957Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.315697] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"867cb10594163f16744de328","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-867cb10594163f16744de328: start","job_status":"start","queue_duration_s":1.465162,"scheduling_latency_s":1.465175} gitlab # [ 167.316444] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.968Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"867cb10594163f16744de328","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-867cb10594163f16744de328: done: 0.009658 sec","job_status":"done","queue_duration_s":1.465162,"scheduling_latency_s":1.465175,"redis_calls":13,"redis_duration_s":0.002389,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000807,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001582,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009658,"completed_at":"2026-07-21T07:46:02.968Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.364723] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.969Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24ded7a7aafad5fa35de5248","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24ded7a7aafad5fa35de5248: start","job_status":"start","queue_duration_s":1.475857,"scheduling_latency_s":1.47587} gitlab # [ 167.365657] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24ded7a7aafad5fa35de5248","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24ded7a7aafad5fa35de5248: done: 0.009653 sec","job_status":"done","queue_duration_s":1.475857,"scheduling_latency_s":1.47587,"redis_calls":13,"redis_duration_s":0.002444,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000795,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001649,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009653,"completed_at":"2026-07-21T07:46:02.978Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.377513] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f778329e7ff2835dc9c2dfdf","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f778329e7ff2835dc9c2dfdf: start","job_status":"start","queue_duration_s":1.486655,"scheduling_latency_s":1.486667} gitlab # [ 167.388264] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f778329e7ff2835dc9c2dfdf","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f778329e7ff2835dc9c2dfdf: done: 0.009664 sec","job_status":"done","queue_duration_s":1.486655,"scheduling_latency_s":1.486667,"redis_calls":13,"redis_duration_s":0.002405,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000765,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00164,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009664,"completed_at":"2026-07-21T07:46:02.989Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.430112] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:02.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cedc9b83dfc2ce6715f3f23","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cedc9b83dfc2ce6715f3f23: start","job_status":"start","queue_duration_s":1.497371,"scheduling_latency_s":1.497384} gitlab # [ 167.433205] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.000Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cedc9b83dfc2ce6715f3f23","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cedc9b83dfc2ce6715f3f23: done: 0.009505 sec","job_status":"done","queue_duration_s":1.497371,"scheduling_latency_s":1.497384,"redis_calls":13,"redis_duration_s":0.002254,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001582,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009505,"completed_at":"2026-07-21T07:46:03.000Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.474754] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.001Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02ed40facbe3106e7e4ef304","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02ed40facbe3106e7e4ef304: start","job_status":"start","queue_duration_s":1.507686,"scheduling_latency_s":1.507699} gitlab # [ 167.478158] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02ed40facbe3106e7e4ef304","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02ed40facbe3106e7e4ef304: done: 0.009135 sec","job_status":"done","queue_duration_s":1.507686,"scheduling_latency_s":1.507699,"redis_calls":13,"redis_duration_s":0.001831,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000658,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001173,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009135,"completed_at":"2026-07-21T07:46:03.010Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.522193] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2507d5f3f9698b8c9e014a7","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2507d5f3f9698b8c9e014a7: start","job_status":"start","queue_duration_s":1.517833,"scheduling_latency_s":1.517845} gitlab # [ 167.523282] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.021Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2507d5f3f9698b8c9e014a7","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2507d5f3f9698b8c9e014a7: done: 0.00972 sec","job_status":"done","queue_duration_s":1.517833,"scheduling_latency_s":1.517845,"redis_calls":13,"redis_duration_s":0.002477,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000813,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001664,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005709,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00972,"completed_at":"2026-07-21T07:46:03.020Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.580633] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.022Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63d37ccd263ba9c028dd6792","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63d37ccd263ba9c028dd6792: start","job_status":"start","queue_duration_s":1.528599,"scheduling_latency_s":1.528612} gitlab # [ 167.581250] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63d37ccd263ba9c028dd6792","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63d37ccd263ba9c028dd6792: done: 0.010161 sec","job_status":"done","queue_duration_s":1.528599,"scheduling_latency_s":1.528612,"redis_calls":13,"redis_duration_s":0.002827,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000774,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010161,"completed_at":"2026-07-21T07:46:03.032Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.634744] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4805944daaeda612e2f7d011","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4805944daaeda612e2f7d011: start","job_status":"start","queue_duration_s":1.539838,"scheduling_latency_s":1.53985} gitlab # [ 167.635514] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.043Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4805944daaeda612e2f7d011","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4805944daaeda612e2f7d011: done: 0.009841 sec","job_status":"done","queue_duration_s":1.539838,"scheduling_latency_s":1.53985,"redis_calls":13,"redis_duration_s":0.002587,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000808,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001779,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009841,"completed_at":"2026-07-21T07:46:03.043Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.685530] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"113d66bde0187a5fe6d67730","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-113d66bde0187a5fe6d67730: start","job_status":"start","queue_duration_s":1.550699,"scheduling_latency_s":1.550712} gitlab # [ 167.686283] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"113d66bde0187a5fe6d67730","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-113d66bde0187a5fe6d67730: done: 0.008868 sec","job_status":"done","queue_duration_s":1.550699,"scheduling_latency_s":1.550712,"redis_calls":13,"redis_duration_s":0.0015919999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005565,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008868,"completed_at":"2026-07-21T07:46:03.052Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.725545] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46429741b3dbb22b00bb45f9","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46429741b3dbb22b00bb45f9: start","job_status":"start","queue_duration_s":1.560376,"scheduling_latency_s":1.560389} gitlab # [ 167.726242] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46429741b3dbb22b00bb45f9","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46429741b3dbb22b00bb45f9: done: 0.00867 sec","job_status":"done","queue_duration_s":1.560376,"scheduling_latency_s":1.560389,"redis_calls":13,"redis_duration_s":0.0015149999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00867,"completed_at":"2026-07-21T07:46:03.062Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.771631] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b932c1f9576cd97d35b6388a","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b932c1f9576cd97d35b6388a: start","job_status":"start","queue_duration_s":1.570001,"scheduling_latency_s":1.570014} gitlab # [ 167.772235] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b932c1f9576cd97d35b6388a","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b932c1f9576cd97d35b6388a: done: 0.009473 sec","job_status":"done","queue_duration_s":1.570001,"scheduling_latency_s":1.570014,"redis_calls":13,"redis_duration_s":0.002334,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001579,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005679,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009473,"completed_at":"2026-07-21T07:46:03.072Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.784907] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6046c3fc472a1f08d2c0ff3","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6046c3fc472a1f08d2c0ff3: start","job_status":"start","queue_duration_s":1.58058,"scheduling_latency_s":1.580592} gitlab # [ 167.796587] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6046c3fc472a1f08d2c0ff3","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6046c3fc472a1f08d2c0ff3: done: 0.009715 sec","job_status":"done","queue_duration_s":1.58058,"scheduling_latency_s":1.580592,"redis_calls":13,"redis_duration_s":0.002467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000849,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001618,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009715,"completed_at":"2026-07-21T07:46:03.083Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.831769] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e22c697d37b8ed2dde00f28","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e22c697d37b8ed2dde00f28: start","job_status":"start","queue_duration_s":1.591361,"scheduling_latency_s":1.591373} gitlab # [ 167.876061] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e22c697d37b8ed2dde00f28","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e22c697d37b8ed2dde00f28: done: 0.009651 sec","job_status":"done","queue_duration_s":1.591361,"scheduling_latency_s":1.591373,"redis_calls":13,"redis_duration_s":0.0025009999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000773,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001728,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009651,"completed_at":"2026-07-21T07:46:03.094Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.877068] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f2390adc570ee0557c5dab1","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f2390adc570ee0557c5dab1: start","job_status":"start","queue_duration_s":1.602131,"scheduling_latency_s":1.602143} gitlab # [ 167.888055] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f2390adc570ee0557c5dab1","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f2390adc570ee0557c5dab1: done: 0.009465 sec","job_status":"done","queue_duration_s":1.602131,"scheduling_latency_s":1.602143,"redis_calls":13,"redis_duration_s":0.002393,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000739,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001654,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009465,"completed_at":"2026-07-21T07:46:03.105Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.933087] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d9d733586089aee2693ff86","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d9d733586089aee2693ff86: start","job_status":"start","queue_duration_s":1.612463,"scheduling_latency_s":1.612476} gitlab # [ 167.933787] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d9d733586089aee2693ff86","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d9d733586089aee2693ff86: done: 0.008777 sec","job_status":"done","queue_duration_s":1.612463,"scheduling_latency_s":1.612476,"redis_calls":13,"redis_duration_s":0.001565,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008777,"completed_at":"2026-07-21T07:46:03.114Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 167.979620] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2393638a4ca7da180d7cb3c","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2393638a4ca7da180d7cb3c: start","job_status":"start","queue_duration_s":1.622031,"scheduling_latency_s":1.622044} gitlab # [ 167.980494] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2393638a4ca7da180d7cb3c","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2393638a4ca7da180d7cb3c: done: 0.008816 sec","job_status":"done","queue_duration_s":1.622031,"scheduling_latency_s":1.622044,"redis_calls":13,"redis_duration_s":0.0014709999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00048,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005464,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008816,"completed_at":"2026-07-21T07:46:03.124Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.022317] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"586603d5cfd7211ea7ae1879","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-586603d5cfd7211ea7ae1879: start","job_status":"start","queue_duration_s":1.63187,"scheduling_latency_s":1.631883} gitlab # [ 168.023706] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"586603d5cfd7211ea7ae1879","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-586603d5cfd7211ea7ae1879: done: 0.008989 sec","job_status":"done","queue_duration_s":1.63187,"scheduling_latency_s":1.631883,"redis_calls":13,"redis_duration_s":0.0016510000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001162,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008989,"completed_at":"2026-07-21T07:46:03.134Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.066206] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79912069656c51bfc6d42bc1","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79912069656c51bfc6d42bc1: start","job_status":"start","queue_duration_s":1.641668,"scheduling_latency_s":1.64168} gitlab # [ 168.068683] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79912069656c51bfc6d42bc1","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79912069656c51bfc6d42bc1: done: 0.011443 sec","job_status":"done","queue_duration_s":1.641668,"scheduling_latency_s":1.64168,"redis_calls":13,"redis_duration_s":0.001774,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000661,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011443,"completed_at":"2026-07-21T07:46:03.146Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.115205] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4860d7144213294ff52d6952","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4860d7144213294ff52d6952: start","job_status":"start","queue_duration_s":1.654084,"scheduling_latency_s":1.654102} gitlab # [ 168.115906] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4860d7144213294ff52d6952","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4860d7144213294ff52d6952: done: 0.009122 sec","job_status":"done","queue_duration_s":1.654084,"scheduling_latency_s":1.654102,"redis_calls":13,"redis_duration_s":0.0014750000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009122,"completed_at":"2026-07-21T07:46:03.156Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.158020] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cccdf0720b8403eb333152a0","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cccdf0720b8403eb333152a0: start","job_status":"start","queue_duration_s":1.66397,"scheduling_latency_s":1.663983} gitlab # [ 168.158675] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cccdf0720b8403eb333152a0","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cccdf0720b8403eb333152a0: done: 0.008975 sec","job_status":"done","queue_duration_s":1.66397,"scheduling_latency_s":1.663983,"redis_calls":13,"redis_duration_s":0.001466,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000466,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008975,"completed_at":"2026-07-21T07:46:03.166Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.197042] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f6706294d70370e1e8d1097","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f6706294d70370e1e8d1097: start","job_status":"start","queue_duration_s":1.673705,"scheduling_latency_s":1.673717} gitlab # [ 168.203229] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f6706294d70370e1e8d1097","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f6706294d70370e1e8d1097: done: 0.008693 sec","job_status":"done","queue_duration_s":1.673705,"scheduling_latency_s":1.673717,"redis_calls":13,"redis_duration_s":0.0015040000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008693,"completed_at":"2026-07-21T07:46:03.175Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.248618] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74c6c07dcbe77d5840865ead","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74c6c07dcbe77d5840865ead: start","job_status":"start","queue_duration_s":1.68331,"scheduling_latency_s":1.683322} gitlab # [ 168.249513] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74c6c07dcbe77d5840865ead","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74c6c07dcbe77d5840865ead: done: 0.008927 sec","job_status":"done","queue_duration_s":1.68331,"scheduling_latency_s":1.683322,"redis_calls":13,"redis_duration_s":0.001607,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001068,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005713,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008927,"completed_at":"2026-07-21T07:46:03.185Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.283658] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91dff55a665fe86ee965986b","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91dff55a665fe86ee965986b: start","job_status":"start","queue_duration_s":1.693032,"scheduling_latency_s":1.693045} gitlab # [ 168.294270] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"91dff55a665fe86ee965986b","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-91dff55a665fe86ee965986b: done: 0.009285 sec","job_status":"done","queue_duration_s":1.693032,"scheduling_latency_s":1.693045,"redis_calls":13,"redis_duration_s":0.001608,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001039,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009285,"completed_at":"2026-07-21T07:46:03.195Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.354709] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.196Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12aefb2e02af8b2c9938b7a3","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12aefb2e02af8b2c9938b7a3: start","job_status":"start","queue_duration_s":1.703404,"scheduling_latency_s":1.703416} gitlab # [ 168.355483] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12aefb2e02af8b2c9938b7a3","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12aefb2e02af8b2c9938b7a3: done: 0.008966 sec","job_status":"done","queue_duration_s":1.703404,"scheduling_latency_s":1.703416,"redis_calls":13,"redis_duration_s":0.001677,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000638,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001039,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008966,"completed_at":"2026-07-21T07:46:03.205Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.374850] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.206Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a89a6455652473258c70a067","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a89a6455652473258c70a067: start","job_status":"start","queue_duration_s":1.713167,"scheduling_latency_s":1.71318} gitlab # [ 168.385947] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a89a6455652473258c70a067","created_at":"2026-07-21T07:46:01.279Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-b60ac9a068964fbc","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"303a80ad1fc5b217979b2243daba0b8a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.493Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a89a6455652473258c70a067: done: 0.008776 sec","job_status":"done","queue_duration_s":1.713167,"scheduling_latency_s":1.71318,"redis_calls":13,"redis_duration_s":0.0015240000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005628,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008776,"completed_at":"2026-07-21T07:46:03.215Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.435647] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.216Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ddc2d0212f0361d3242bea49","created_at":"2026-07-21T07:46:01.499Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0430d6ecd098e1742e32a92f993544a0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.502Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ddc2d0212f0361d3242bea49: start","job_status":"start","queue_duration_s":1.713819,"scheduling_latency_s":1.713832} gitlab # [ 168.436611] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.224Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ddc2d0212f0361d3242bea49","created_at":"2026-07-21T07:46:01.499Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0430d6ecd098e1742e32a92f993544a0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.502Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ddc2d0212f0361d3242bea49: done: 0.008066 sec","job_status":"done","queue_duration_s":1.713819,"scheduling_latency_s":1.713832,"redis_calls":8,"redis_duration_s":0.00099,"redis_read_bytes":133,"redis_write_bytes":901,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000123,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000622,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000245,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005169,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008066,"completed_at":"2026-07-21T07:46:03.224Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.475496] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.225Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7d22c0d128dbf3d8cea536a3","created_at":"2026-07-21T07:46:01.510Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dddf816bb17dbf0c6006cf6dacbcf290","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.578Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7d22c0d128dbf3d8cea536a3: start","job_status":"start","queue_duration_s":1.646091,"scheduling_latency_s":1.646103} gitlab # [ 168.514519] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.229Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 168.515105] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.232Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7d22c0d128dbf3d8cea536a3","created_at":"2026-07-21T07:46:01.510Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dddf816bb17dbf0c6006cf6dacbcf290","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.578Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7d22c0d128dbf3d8cea536a3: done: 0.007637 sec","job_status":"done","queue_duration_s":1.646091,"scheduling_latency_s":1.646103,"redis_calls":7,"redis_duration_s":0.0010299999999999999,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007637,"completed_at":"2026-07-21T07:46:03.232Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.557612] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.233Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1f56ee641aaab2051a4959e4","created_at":"2026-07-21T07:46:01.591Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0266105f3eb09537588b25c997475887","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.595Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1f56ee641aaab2051a4959e4: start","job_status":"start","queue_duration_s":1.638258,"scheduling_latency_s":1.638269} gitlab # [ 168.558222] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.243Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1f56ee641aaab2051a4959e4","created_at":"2026-07-21T07:46:01.591Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0266105f3eb09537588b25c997475887","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.595Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1f56ee641aaab2051a4959e4: done: 0.010355 sec","job_status":"done","queue_duration_s":1.638258,"scheduling_latency_s":1.638269,"redis_calls":9,"redis_duration_s":0.001353,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000683,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00067,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000314845,"exclusive_lock_hold_duration_s":0.002118985,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.010355,"completed_at":"2026-07-21T07:46:03.243Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000914,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.604553] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.244Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"cc6d11da7d01c31a4634db1b","created_at":"2026-07-21T07:46:01.603Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ffb759e85429dcfa53b14583fb689044","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-21T07:46:01.605Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-cc6d11da7d01c31a4634db1b: start","job_status":"start","queue_duration_s":1.639177,"scheduling_latency_s":1.639189} gitlab # [ 168.605264] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.257Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"cc6d11da7d01c31a4634db1b","created_at":"2026-07-21T07:46:01.603Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ffb759e85429dcfa53b14583fb689044","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-07-21T07:46:01.605Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-cc6d11da7d01c31a4634db1b: done: 0.013062 sec","job_status":"done","queue_duration_s":1.639177,"scheduling_latency_s":1.639189,"redis_calls":7,"redis_duration_s":0.00101,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000765,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000245,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013062,"completed_at":"2026-07-21T07:46:03.257Z","load_balancing_strategy":"primary","db_duration_s":0.000964,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.615599] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.258Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"7cb167c9d758845c1de4603d","created_at":"2026-07-21T07:46:01.614Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6cb23cd7c4063265c68475ad0747e563","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-21T07:46:01.616Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-7cb167c9d758845c1de4603d: start","job_status":"start","queue_duration_s":1.642274,"scheduling_latency_s":1.642287} gitlab # [ 168.661829] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.263Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 168.662582] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"7cb167c9d758845c1de4603d","created_at":"2026-07-21T07:46:01.614Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6cb23cd7c4063265c68475ad0747e563","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-07-21T07:46:01.616Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-7cb167c9d758845c1de4603d: done: 0.007671 sec","job_status":"done","queue_duration_s":1.642274,"scheduling_latency_s":1.642287,"redis_calls":7,"redis_duration_s":0.000889,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000647,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007671,"completed_at":"2026-07-21T07:46:03.266Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.663954] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.267Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"93639e8d50eb88538b3c6e6d","created_at":"2026-07-21T07:46:01.625Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4015d64ed9b816082f14c3eb21097389","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.628Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-93639e8d50eb88538b3c6e6d: start","job_status":"start","queue_duration_s":1.638332,"scheduling_latency_s":1.638345} gitlab # [ 168.676807] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.278Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"93639e8d50eb88538b3c6e6d","created_at":"2026-07-21T07:46:01.625Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4015d64ed9b816082f14c3eb21097389","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.628Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-93639e8d50eb88538b3c6e6d: done: 0.011143 sec","job_status":"done","queue_duration_s":1.638332,"scheduling_latency_s":1.638345,"redis_calls":7,"redis_duration_s":0.000938,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000697,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011143,"completed_at":"2026-07-21T07:46:03.278Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002015,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.743830] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.279Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1df7ad2bfba6ba887275ec5b","created_at":"2026-07-21T07:46:01.633Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3bd39f421072384391ad1f5f1cdcdc11","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-21T07:46:01.636Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1df7ad2bfba6ba887275ec5b: start","job_status":"start","queue_duration_s":1.642464,"scheduling_latency_s":1.642477} gitlab # [ 168.744373] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.283Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 168.744907] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.286Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"1df7ad2bfba6ba887275ec5b","created_at":"2026-07-21T07:46:01.633Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3bd39f421072384391ad1f5f1cdcdc11","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-07-21T07:46:01.636Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-1df7ad2bfba6ba887275ec5b: done: 0.007684 sec","job_status":"done","queue_duration_s":1.642464,"scheduling_latency_s":1.642477,"redis_calls":7,"redis_duration_s":0.000937,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000695,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00483,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007684,"completed_at":"2026-07-21T07:46:03.286Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.794707] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.287Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"aacc2c24102a59957ff0789d","created_at":"2026-07-21T07:46:01.688Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a264157a742aa54ba9a3feee599ea751","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.715Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-aacc2c24102a59957ff0789d: start","job_status":"start","queue_duration_s":1.571659,"scheduling_latency_s":1.571672} gitlab # [ 168.795441] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.305Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"aacc2c24102a59957ff0789d","created_at":"2026-07-21T07:46:01.688Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a264157a742aa54ba9a3feee599ea751","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.715Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-aacc2c24102a59957ff0789d: done: 0.017783 sec","job_status":"done","queue_duration_s":1.571659,"scheduling_latency_s":1.571672,"redis_calls":10,"redis_duration_s":0.001482,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000355,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000854,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000273,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00020673,"duration_s":0.017783,"completed_at":"2026-07-21T07:46:03.305Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002092,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.810667] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.306Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4497e7a866f7185431c4ce74","created_at":"2026-07-21T07:46:01.725Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f876f65e26102497abda2f471f0d7df1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.728Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4497e7a866f7185431c4ce74: start","job_status":"start","queue_duration_s":1.577472,"scheduling_latency_s":1.577485} gitlab # [ 168.823124] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.314Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"4497e7a866f7185431c4ce74","created_at":"2026-07-21T07:46:01.725Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f876f65e26102497abda2f471f0d7df1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.728Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-4497e7a866f7185431c4ce74: done: 0.007988 sec","job_status":"done","queue_duration_s":1.577472,"scheduling_latency_s":1.577485,"redis_calls":7,"redis_duration_s":0.0008730000000000001,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000608,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007988,"completed_at":"2026-07-21T07:46:03.314Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.888535] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.315Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f7c90fdc09b72b0f01b0c32e","created_at":"2026-07-21T07:46:01.743Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a26096081f5671135d34880a9291d307","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.747Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f7c90fdc09b72b0f01b0c32e: start","job_status":"start","queue_duration_s":1.567251,"scheduling_latency_s":1.567263} gitlab # [ 168.889193] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.320Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 168.889715] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.323Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f7c90fdc09b72b0f01b0c32e","created_at":"2026-07-21T07:46:01.743Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a26096081f5671135d34880a9291d307","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.747Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f7c90fdc09b72b0f01b0c32e: done: 0.008527 sec","job_status":"done","queue_duration_s":1.567251,"scheduling_latency_s":1.567263,"redis_calls":7,"redis_duration_s":0.001094,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000822,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008527,"completed_at":"2026-07-21T07:46:03.323Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.910746] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.324Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"2b47b2407a3d9e9b20f2e3b1","created_at":"2026-07-21T07:46:01.805Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d336f782e1aebfdb0ad9b07a49e6cef","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.884Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-2b47b2407a3d9e9b20f2e3b1: start","job_status":"start","queue_duration_s":1.44036,"scheduling_latency_s":1.440374} gitlab # [ 168.923023] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.338Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"2b47b2407a3d9e9b20f2e3b1","created_at":"2026-07-21T07:46:01.805Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-2b9d8570fe3f4c41","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d336f782e1aebfdb0ad9b07a49e6cef","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:01.884Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-2b47b2407a3d9e9b20f2e3b1: done: 0.013559 sec","job_status":"done","queue_duration_s":1.44036,"scheduling_latency_s":1.440374,"redis_calls":7,"redis_duration_s":0.001067,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.013559,"completed_at":"2026-07-21T07:46:03.338Z","load_balancing_strategy":"primary","db_duration_s":0.004454,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 168.961910] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.338Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1e4b3d7e5ada6d6417dfc752","created_at":"2026-07-21T07:46:02.436Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-66a2843cab1f4b79","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"817362336569c4ccedc93068ebef469d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:02.448Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1e4b3d7e5ada6d6417dfc752: start","job_status":"start","queue_duration_s":0.890253,"scheduling_latency_s":0.890266} gitlab # [ 168.969127] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.345Z","message":"No existing merge request to be cleaned up."} gitlab # [ 168.970580] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.351Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1e4b3d7e5ada6d6417dfc752","created_at":"2026-07-21T07:46:02.436Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-66a2843cab1f4b79","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"817362336569c4ccedc93068ebef469d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:02.448Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1e4b3d7e5ada6d6417dfc752: done: 0.012075 sec","job_status":"done","queue_duration_s":0.890253,"scheduling_latency_s":0.890266,"redis_calls":10,"redis_duration_s":0.001411,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012075,"completed_at":"2026-07-21T07:46:03.351Z","load_balancing_strategy":"primary","db_duration_s":0.001338,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.023982] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.352Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5f4f0c6de2a7dfb49028623d","created_at":"2026-07-21T07:46:02.436Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-66a2843cab1f4b79","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"817362336569c4ccedc93068ebef469d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:02.448Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5f4f0c6de2a7dfb49028623d: start","job_status":"start","queue_duration_s":0.903418,"scheduling_latency_s":0.903433} gitlab # [ 169.024658] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.362Z","message":"No existing merge request to be cleaned up."} gitlab # [ 169.025213] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.367Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5f4f0c6de2a7dfb49028623d","created_at":"2026-07-21T07:46:02.436Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-66a2843cab1f4b79","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"817362336569c4ccedc93068ebef469d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:02.448Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5f4f0c6de2a7dfb49028623d: done: 0.015753 sec","job_status":"done","queue_duration_s":0.903418,"scheduling_latency_s":0.903433,"redis_calls":10,"redis_duration_s":0.001373,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000831,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015753,"completed_at":"2026-07-21T07:46:03.367Z","load_balancing_strategy":"primary","db_duration_s":0.008654,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.086745] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.368Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1bfcba3efb97c8797ba875a9","created_at":"2026-07-21T07:46:02.436Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-66a2843cab1f4b79","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"817362336569c4ccedc93068ebef469d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:02.448Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1bfcba3efb97c8797ba875a9: start","job_status":"start","queue_duration_s":0.920063,"scheduling_latency_s":0.920078} gitlab # [ 169.087457] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.374Z","message":"No existing merge request to be cleaned up."} gitlab # [ 169.087943] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.379Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1bfcba3efb97c8797ba875a9","created_at":"2026-07-21T07:46:02.436Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-66a2843cab1f4b79","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"817362336569c4ccedc93068ebef469d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:02.448Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1bfcba3efb97c8797ba875a9: done: 0.010354 sec","job_status":"done","queue_duration_s":0.920063,"scheduling_latency_s":0.920078,"redis_calls":10,"redis_duration_s":0.0013,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000786,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00677,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010354,"completed_at":"2026-07-21T07:46:03.379Z","load_balancing_strategy":"primary","db_duration_s":0.000876,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 169.095275] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.379Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3f22d56313b279bc8a13137d","created_at":"2026-07-21T07:46:02.436Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-66a2843cab1f4b79","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"817362336569c4ccedc93068ebef469d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:02.448Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3f22d56313b279bc8a13137d: start","job_status":"start","queue_duration_s":0.931211,"scheduling_latency_s":0.931225} gitlab # [ 169.106129] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.385Z","message":"No existing merge request to be cleaned up."} gitlab # [ 169.107697] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:03.389Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3f22d56313b279bc8a13137d","created_at":"2026-07-21T07:46:02.436Z","trace_propagation_headers":{"sentry-trace":"200cc609d9814e34b19043eeb87b5ded-66a2843cab1f4b79","baggage":"sentry-trace_id=200cc609d9814e34b19043eeb87b5ded,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"817362336569c4ccedc93068ebef469d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:02.448Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3f22d56313b279bc8a13137d: done: 0.010023 sec","job_status":"done","queue_duration_s":0.931211,"scheduling_latency_s":0.931225,"redis_calls":10,"redis_duration_s":0.001295,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000753,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010023,"completed_at":"2026-07-21T07:46:03.389Z","load_balancing_strategy":"primary","db_duration_s":0.000868,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.608738] runuser[2608]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 175.613714] setup-gitlab-start[2803]: ++ jq -r .token gitlab # [ 175.615102] setup-gitlab-start[2804]: +++ date -d '+7 days' +%F gitlab # [ 175.616742] setup-gitlab-start[2802]: ++ curl -fs -X POST -H 'Authorization: Bearer 41c843e4023b9531ffbba8bafe664da433f9ff860e7ff8e30ee758b488c40e28' -H 'Content-Type: application/json' -d '{"name":"nixbot","scopes":["api"],"expires_at":"2026-07-28"}' http://gitlab/api/v4/users/1/personal_access_tokens gitlab # [ 175.764183] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:12.454Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"20541a141874b4d2c9494f49","created_at":"2026-07-21T07:46:12.433Z","trace_propagation_headers":{"sentry-trace":"2773c7069d6e4f27891c8cba669e545d-69fb526822224d68","baggage":"sentry-trace_id=2773c7069d6e4f27891c8cba669e545d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7DPVQVBWD37KZXSQBRNG","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:b358c01956859d066a7f63e84c44f0e1c2814fff887c9f336397adfc69bb129d","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:12.451Z","job_size_bytes":451,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-20541a141874b4d2c9494f49: start","job_status":"start","queue_duration_s":0.002505,"scheduling_latency_s":0.002552} gitlab # [ 175.767495] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:12 +0000] "POST /api/v4/users/1/personal_access_tokens HTTP/1.1" 201 267 "" "curl/8.20.0" 141 gitlab # [ 175.804291] setup-gitlab-start[2566]: + TOKEN=glpat-rUMWaFKgrBdeuBsxSLHd3G86MQp1OjEH.01.0w056qjbu gitlab # [ 175.805382] setup-gitlab-start[2566]: + test -n glpat-rUMWaFKgrBdeuBsxSLHd3G86MQp1OjEH.01.0w056qjbu gitlab # [ 175.806461] setup-gitlab-start[2566]: + test glpat-rUMWaFKgrBdeuBsxSLHd3G86MQp1OjEH.01.0w056qjbu '!=' null gitlab # [ 175.807952] setup-gitlab-start[2566]: + mkdir -p /var/lib/secrets gitlab # [ 175.816795] setup-gitlab-start[2566]: + echo glpat-rUMWaFKgrBdeuBsxSLHd3G86MQp1OjEH.01.0w056qjbu gitlab # [ 175.819617] setup-gitlab-start[2566]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 175.822080] setup-gitlab-start[2566]: + curl -fs -X POST -H 'PRIVATE-TOKEN: glpat-rUMWaFKgrBdeuBsxSLHd3G86MQp1OjEH.01.0w056qjbu' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects gitlab # [ 175.879194] sidekiq-cluster[2039]: {"severity":"WARN","time":"2026-07-21T07:46:12.569Z","retry":3,"queue":"mailers","store":null,"args":["[FILTERED]"],"class":"ActionMailer::MailDeliveryJob","jid":"20541a141874b4d2c9494f49","created_at":"2026-07-21T07:46:12.433Z","trace_propagation_headers":{"sentry-trace":"2773c7069d6e4f27891c8cba669e545d-69fb526822224d68","baggage":"sentry-trace_id=2773c7069d6e4f27891c8cba669e545d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7DPVQVBWD37KZXSQBRNG","meta.caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"system_access","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/users/:user_id/personal_access_tokens","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:mailers:b358c01956859d066a7f63e84c44f0e1c2814fff887c9f336397adfc69bb129d","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:12.451Z","job_size_bytes":451,"pid":2039,"sidekiq_tid":"m0p3","sidekiq_thread_name":"sidekiq.default/processor","message":"ActionMailer::MailDeliveryJob JID-20541a141874b4d2c9494f49: fail: 0.115544 sec","job_status":"fail","queue_duration_s":0.002505,"scheduling_latency_s":0.002552,"redis_calls":11,"redis_duration_s":0.002664,"redis_read_bytes":137,"redis_write_bytes":1317,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000864,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001333,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.082427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000226286,"duration_s":0.115544,"completed_at":"2026-07-21T07:46:12.569Z","load_balancing_strategy":"primary_no_wal","exception.class":"ArgumentError","exception.message":":arguments expected to be an Array of individual string args","exception.backtrace":["/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/network/delivery_methods/sendmail.rb:53:in `initialize'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `new'","/nix/store/h836fb4ajnymmg0ash24wg1cb4qbhnpd-ruby3.3-mail-2.9.0/lib/ruby/gems/3.3.0/gems/mail-2.9.0/lib/mail/message.rb:278:in `delivery_method'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:66:in `wrap_delivery_behavior'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/delivery_methods.rb:80:in `wrap_delivery_behavior!'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:876:in `mail'","app/mailers/application_mailer.rb:85:in `block in mail_with_locale'","lib/gitlab/i18n.rb:116:in `with_locale'","app/mailers/application_mailer.rb:84:in `mail_with_locale'","app/mailers/notify.rb:293:in `email_with_layout'","app/mailers/emails/profile.rb:99:in `access_token_created_email'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:215:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:261:in `block in process_action'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","lib/gitlab/i18n.rb:116:in `with_locale'","lib/gitlab/i18n.rb:126:in `with_default_locale'","app/mailers/application_mailer.rb:60:in `render_with_default_locale'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/callbacks.rb:260:in `process_action'","/nix/store/hapk8srbrjgzcpyn7snxh1713ans9a3k-ruby3.3-actionpack-7.2.3/lib/ruby/gems/3.3.0/gems/actionpack-7.2.3/lib/abstract_controller/base.rb:152:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:29:in `block in process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:21:in `handle_exceptions'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/rescuable.rb:28:in `process'","/nix/store/abbaw4w9gll6p923md75p3h3rrmhrdqn-ruby3.3-actionview-7.2.3/lib/ruby/gems/3.3.0/gems/actionview-7.2.3/lib/action_view/rendering.rb:40:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:652:in `block in process'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/base.rb:651:in `process'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:136:in `block in processed_mailer'","\u003cinternal:kernel\u003e:90:in `tap'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:135:in `processed_mailer'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/message_delivery.rb:124:in `deliver_now'","/nix/store/z2822z98p3wywq7nq70f3l458wb38x4k-ruby3.3-actionmailer-7.2.3/lib/ruby/gems/3.3.0/gems/actionmailer-7.2.3/lib/action_mailer/mail_delivery_job.rb:28:in `perform'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:68:in `block in _perform_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/railtie.rb:32:in `block (2 levels) in insert_into_active_job'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/bqz31s23z0jk4464lbybyz83ikl0gmyd-ruby3.3-i18n-1.14.7/lib/ruby/gems/3.3.0/gems/i18n-1.14.7/lib/i18n.rb:353:in `with_locale'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/translation.rb:9:in `block (2 levels) in \u003cmodule:Translation\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/core_ext/time/zones.rb:65:in `use_zone'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/timezones.rb:9:in `block (2 levels) in \u003cmodule:Timezones\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:67:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:32:in `_perform_job'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:51:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `block in perform_now'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:13:in `block in instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:40:in `block in instrument'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/notifications.rb:212:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:39:in `instrument'","/nix/store/l71g3rmwn8nif8ik503h15pklil3njdz-ruby3.3-activerecord-7.2.3/lib/ruby/gems/3.3.0/gems/activerecord-7.2.3/lib/active_record/railties/job_runtime.rb:11:in `instrument'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/instrumentation.rb:26:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `block in perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:41:in `tag_logger'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/logging.rb:32:in `perform_now'","/nix/store/wp2rnxa243q2gn2f8h04d48hnvfr0387-ruby3.3-sentry-rails-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-rails-5.23.0/lib/sentry/rails/active_job.rb:8:in `perform_now'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:29:in `block in execute'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:121:in `block in run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:79:in `block (4 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/railtie.rb:78:in `block (3 levels) in \u003cclass:Railtie\u003e'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `instance_exec'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:130:in `block in run_callbacks'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/callbacks.rb:141:in `run_callbacks'","/nix/store/i614i7jgbmq8nfpazmpxhvhawszwyg1f-ruby3.3-activejob-7.2.3/lib/ruby/gems/3.3.0/gems/activejob-7.2.3/lib/active_job/execution.rb:27:in `execute'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:13:in `perform'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/throttling/middleware.rb:18:in `perform'","lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform'","lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:185:in `block in use'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","lib/gitlab/application_context.rb:185:in `use'","lib/gitlab/application_context.rb:105:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:94:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:10:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:141:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:22:in `block in call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/context.rb:43:in `with_context'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/context/server.rb:15:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/d2b87xpm5ksd634qirnkzkp7bav928v9-ruby3.3-gitlab-labkit-1.5.1/lib/ruby/gems/3.3.0/gems/gitlab-labkit-1.5.1/lib/labkit/middleware/sidekiq/server.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/avw1z19zhyfbsdkh6mbsdbsv02mpq0k9-ruby3.3-marginalia-1.11.1/lib/ruby/gems/3.3.0/gems/marginalia-1.11.1/lib/marginalia/sidekiq_instrumentation.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/85h4805ad7n5qn7wq2m6icypga7rnk80-ruby3.3-sentry-sidekiq-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-sidekiq-5.23.0/lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:77:in `block in wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/execution_wrapper.rb:91:in `wrap'","/nix/store/2cs1nrzqirc1nwy2h32akwq1w5899v7d-ruby3.3-activesupport-7.2.3/lib/ruby/gems/3.3.0/gems/activesupport-7.2.3/lib/active_support/reloader.rb:74:in `wrap'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog'","/nix/store/6b5b40g0i1g6c6ymmg8fd6c8cjf4nh5x-gemfile-and-lockfile/vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread'"],"db_duration_s":0.022482,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.750758] gitaly[1358]: time="2026-07-21T07:46:13.441Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T07:46:23.000" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.441" pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 176.751178] gitaly[1358]: time="2026-07-21T07:46:13.441Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T07:46:23.000" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.441" grpc.time_ms=0.295 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 176.826930] gitaly[1358]: time="2026-07-21T07:46:13.518Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T07:46:23.077" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.517" pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 176.828169] gitaly[1358]: time="2026-07-21T07:46:13.518Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T07:46:23.077" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.517" grpc.time_ms=0.319 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 176.863249] gitaly[1358]: time="2026-07-21T07:46:13.554Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 diskcache=244c6ee4-4ebb-42eb-a2ad-00ad5a7181fc grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-21T07:46:23.082" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.523" pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 176.864893] gitaly[1358]: time="2026-07-21T07:46:13.554Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=321 command.oublock=32 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-21T07:46:23.082" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.523" grpc.time_ms=31.341 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 176.898064] gitaly[1358]: time="2026-07-21T07:46:13.562Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T07:46:23.121" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.561" pid=1358 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 176.898635] gitaly[1358]: time="2026-07-21T07:46:13.589Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T07:46:23.121" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.561" grpc.time_ms=27.456 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 176.906045] gitaly[1358]: time="2026-07-21T07:46:13.597Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-21T07:46:23.152" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.592" grpc.time_ms=4.784 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 176.946508] gitaly[1358]: time="2026-07-21T07:46:13.637Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T07:46:23.197" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.637" pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 176.956311] gitaly[1358]: time="2026-07-21T07:46:13.637Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T07:46:23.197" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.637" grpc.time_ms=0.194 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 176.974961] gitaly[1358]: time="2026-07-21T07:46:13.666Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 diskcache=ce7f189f-a70a-4b69-b2b9-27480015c243 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-21T07:46:23.198" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.639" pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 176.984319] gitaly[1358]: time="2026-07-21T07:46:13.675Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=323 command.oublock=32 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-07-21T07:46:23.198" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.639" grpc.time_ms=27.007 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 177.000770] gitaly[1358]: time="2026-07-21T07:46:13.691Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T07:46:23.251" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.691" pid=1358 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 177.011235] gitaly[1358]: time="2026-07-21T07:46:13.692Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T07:46:23.251" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.691" grpc.time_ms=0.63 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 177.012232] gitaly[1358]: time="2026-07-21T07:46:13.703Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-21T07:46:23.255" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.695" grpc.time_ms=7.442 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 177.025338] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:13.715Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"1ca551721e4dc5e23cdccbf6","created_at":"2026-07-21T07:46:13.603Z","trace_propagation_headers":{"sentry-trace":"c8a8d2e1b58b482f81162541a891e2f7-b726b0dbd4ad4046","baggage":"sentry-trace_id=c8a8d2e1b58b482f81162541a891e2f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7DXDNVPH29WE7TFE89D9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:13.623Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-1ca551721e4dc5e23cdccbf6: start","job_status":"start","queue_duration_s":0.09157,"scheduling_latency_s":0.091602} gitlab # [ 177.035987] gitaly[1358]: time="2026-07-21T07:46:13.727Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T07:46:23.286" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.727" grpc.time_ms=0.126 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 177.040048] gitaly[1358]: time="2026-07-21T07:46:13.731Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-21T07:46:23.000" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:13.731" grpc.time_ms=0.167 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 177.067703] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:13.758Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"1ca551721e4dc5e23cdccbf6","created_at":"2026-07-21T07:46:13.603Z","trace_propagation_headers":{"sentry-trace":"c8a8d2e1b58b482f81162541a891e2f7-b726b0dbd4ad4046","baggage":"sentry-trace_id=c8a8d2e1b58b482f81162541a891e2f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7DXDNVPH29WE7TFE89D9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:13.623Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-1ca551721e4dc5e23cdccbf6: done: 0.043346 sec","job_status":"done","queue_duration_s":0.09157,"scheduling_latency_s":0.091602,"redis_calls":10,"redis_duration_s":0.002332,"redis_read_bytes":8,"redis_write_bytes":1165,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001374,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000958,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":7,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026535,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.023471698,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.043346,"completed_at":"2026-07-21T07:46:13.758Z","load_balancing_strategy":"primary","db_duration_s":0.008657,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 177.106324] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:13.797Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"5053dcf6805094af15737939","created_at":"2026-07-21T07:46:13.631Z","trace_propagation_headers":{"sentry-trace":"c8a8d2e1b58b482f81162541a891e2f7-b726b0dbd4ad4046","baggage":"sentry-trace_id=c8a8d2e1b58b482f81162541a891e2f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7DXDNVPH29WE7TFE89D9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:13.634Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-5053dcf6805094af15737939: start","job_status":"start","queue_duration_s":0.162735,"scheduling_latency_s":0.162758} gitlab # [ 177.196489] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:13.887Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"5053dcf6805094af15737939","created_at":"2026-07-21T07:46:13.631Z","trace_propagation_headers":{"sentry-trace":"c8a8d2e1b58b482f81162541a891e2f7-b726b0dbd4ad4046","baggage":"sentry-trace_id=c8a8d2e1b58b482f81162541a891e2f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7DXDNVPH29WE7TFE89D9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:13.634Z","job_size_bytes":2,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-5053dcf6805094af15737939: done: 0.090178 sec","job_status":"done","queue_duration_s":0.162735,"scheduling_latency_s":0.162758,"redis_calls":14,"redis_duration_s":0.003182,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000406,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001895,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000881,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.058785125,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.090178,"completed_at":"2026-07-21T07:46:13.887Z","load_balancing_strategy":"primary","db_duration_s":0.022934,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 177.349856] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:14.040Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"0a2bacc4f9d66642d00c07d2","created_at":"2026-07-21T07:46:14.034Z","trace_propagation_headers":{"sentry-trace":"c8a8d2e1b58b482f81162541a891e2f7-b726b0dbd4ad4046","baggage":"sentry-trace_id=c8a8d2e1b58b482f81162541a891e2f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7DXDNVPH29WE7TFE89D9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:14.039Z","job_size_bytes":5,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-0a2bacc4f9d66642d00c07d2: start","job_status":"start","queue_duration_s":0.001235,"scheduling_latency_s":0.001265} gitlab # [ 177.459996] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:14.150Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"0a2bacc4f9d66642d00c07d2","created_at":"2026-07-21T07:46:14.034Z","trace_propagation_headers":{"sentry-trace":"c8a8d2e1b58b482f81162541a891e2f7-b726b0dbd4ad4046","baggage":"sentry-trace_id=c8a8d2e1b58b482f81162541a891e2f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7DXDNVPH29WE7TFE89D9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:14.039Z","job_size_bytes":5,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-0a2bacc4f9d66642d00c07d2: done: 0.109582 sec","job_status":"done","queue_duration_s":0.001235,"scheduling_latency_s":0.001265,"redis_calls":17,"redis_duration_s":0.005305,"redis_read_bytes":144,"redis_write_bytes":3399,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.001465,"redis_feature_flag_read_bytes":132,"redis_feature_flag_write_bytes":884,"redis_queues_calls":2,"redis_queues_duration_s":0.000405,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1257,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.003017,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":992,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000418,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":16,"db_write_count":1,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00057633,"duration_s":0.109582,"completed_at":"2026-07-21T07:46:14.150Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021304,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 177.525473] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:14.216Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"77700821b96ca87986864586","created_at":"2026-07-21T07:46:14.114Z","trace_propagation_headers":{"sentry-trace":"c8a8d2e1b58b482f81162541a891e2f7-b726b0dbd4ad4046","baggage":"sentry-trace_id=c8a8d2e1b58b482f81162541a891e2f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7DXDNVPH29WE7TFE89D9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:14.121Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-77700821b96ca87986864586: start","job_status":"start","queue_duration_s":0.094937,"scheduling_latency_s":0.09498} gitlab # [ 177.539292] gitaly[1358]: time="2026-07-21T07:46:14.225Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7DXDNVPH29WE7TFE89D9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-21T07:46:24.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:14.217" grpc.time_ms=8.02 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 177.758910] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:14.449Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"77700821b96ca87986864586","created_at":"2026-07-21T07:46:14.114Z","trace_propagation_headers":{"sentry-trace":"c8a8d2e1b58b482f81162541a891e2f7-b726b0dbd4ad4046","baggage":"sentry-trace_id=c8a8d2e1b58b482f81162541a891e2f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7DXDNVPH29WE7TFE89D9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:14.121Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-77700821b96ca87986864586: done: 0.233492 sec","job_status":"done","queue_duration_s":0.094937,"scheduling_latency_s":0.09498,"redis_calls":11,"redis_duration_s":0.002809,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000603,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001593,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000613,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.177,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.177,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.145755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038301,"duration_s":0.233492,"completed_at":"2026-07-21T07:46:14.449Z","load_balancing_strategy":"primary","db_duration_s":0.081211,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.800590] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:14.491Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"a9155f5fc2f933dd3d22e244","created_at":"2026-07-21T07:46:14.125Z","trace_propagation_headers":{"sentry-trace":"c8a8d2e1b58b482f81162541a891e2f7-b726b0dbd4ad4046","baggage":"sentry-trace_id=c8a8d2e1b58b482f81162541a891e2f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7DXDNVPH29WE7TFE89D9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:14.131Z","job_size_bytes":89,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-a9155f5fc2f933dd3d22e244: start","job_status":"start","queue_duration_s":0.359679,"scheduling_latency_s":0.359701} gitlab # [ 177.892761] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:14.583Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"a9155f5fc2f933dd3d22e244","created_at":"2026-07-21T07:46:14.125Z","trace_propagation_headers":{"sentry-trace":"c8a8d2e1b58b482f81162541a891e2f7-b726b0dbd4ad4046","baggage":"sentry-trace_id=c8a8d2e1b58b482f81162541a891e2f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7DXDNVPH29WE7TFE89D9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:14.131Z","job_size_bytes":89,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-a9155f5fc2f933dd3d22e244: done: 0.091764 sec","job_status":"done","queue_duration_s":0.359679,"scheduling_latency_s":0.359701,"redis_calls":11,"redis_duration_s":0.003432,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000876,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00186,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000696,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000541689,"duration_s":0.091764,"completed_at":"2026-07-21T07:46:14.583Z","load_balancing_strategy":"primary","db_duration_s":0.026898,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 177.956721] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:14.647Z","retry":25,"queue":"default","version":0,"store":null,"class":"Ci::PipelineSchedules::DeactivateSchedulesWorker","args":["ProjectAuthorizations::AuthorizationsAddedEvent","[{\"project_ids\"=\u003e[1], \"user_ids\"=\u003e[1]}]"],"jid":"7863b1650a9c0010b43b1620","created_at":"2026-07-21T07:46:14.128Z","trace_propagation_headers":{"sentry-trace":"c8a8d2e1b58b482f81162541a891e2f7-54661cd3ae6e4d3f","baggage":"sentry-trace_id=c8a8d2e1b58b482f81162541a891e2f7,sentry-release=Unknown"},"correlation_id":"01KY1T7DXDNVPH29WE7TFE89D9","meta.caller_id":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:39ab203788ffde42edba0d838663a2477aee99a8f3196d11bf650021a1ee799d","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:14.140Z","job_size_bytes":88,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineSchedules::DeactivateSchedulesWorker JID-7863b1650a9c0010b43b1620: start","job_status":"start","queue_duration_s":0.506425,"scheduling_latency_s":0.506525} gitlab # [ 177.971181] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:14 +0000] "POST /api/v4/projects HTTP/1.1" 201 4795 "" "curl/8.20.0" 2136 gitlab # [ 177.974976] setup-gitlab-start[2566]: + mkdir -p /tmp/test-flake gitlab # [ 177.977929] setup-gitlab-start[2566]: + cd /tmp/test-flake gitlab # [ 177.978681] setup-gitlab-start[2566]: + git init -b master gitlab # [ 177.999969] setup-gitlab-start[2847]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 178.000489] setup-gitlab-start[2566]: + git config user.name test gitlab # [ 178.005096] setup-gitlab-start[2566]: + git config user.email test@example.com gitlab # [ 178.008229] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:14.699Z","retry":25,"queue":"default","version":0,"store":null,"class":"Ci::PipelineSchedules::DeactivateSchedulesWorker","args":["ProjectAuthorizations::AuthorizationsAddedEvent","[{\"project_ids\"=\u003e[1], \"user_ids\"=\u003e[1]}]"],"jid":"7863b1650a9c0010b43b1620","created_at":"2026-07-21T07:46:14.128Z","trace_propagation_headers":{"sentry-trace":"c8a8d2e1b58b482f81162541a891e2f7-54661cd3ae6e4d3f","baggage":"sentry-trace_id=c8a8d2e1b58b482f81162541a891e2f7,sentry-release=Unknown"},"correlation_id":"01KY1T7DXDNVPH29WE7TFE89D9","meta.caller_id":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:39ab203788ffde42edba0d838663a2477aee99a8f3196d11bf650021a1ee799d","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:14.140Z","job_size_bytes":88,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineSchedules::DeactivateSchedulesWorker JID-7863b1650a9c0010b43b1620: done: 0.051943 sec","job_status":"done","queue_duration_s":0.506425,"scheduling_latency_s":0.506525,"redis_calls":10,"redis_duration_s":0.003043,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000402,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001229,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001412,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000225168,"duration_s":0.051943,"completed_at":"2026-07-21T07:46:14.699Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005023,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 178.009778] setup-gitlab-start[2566]: + cat gitlab # [ 178.011380] setup-gitlab-start[2566]: + git add flake.nix gitlab # [ 178.013642] setup-gitlab-start[2566]: + git commit -m 'initial commit' gitlab # [ 178.019478] setup-gitlab-start[2852]: [master (root-commit) 7e3d3d4] initial commit gitlab # [ 178.020373] setup-gitlab-start[2852]: 1 file changed, 10 insertions(+) gitlab # [ 178.021021] setup-gitlab-start[2852]: create mode 100644 flake.nix gitlab # [ 178.021498] setup-gitlab-start[2566]: + git remote add origin http://oauth2:glpat-rUMWaFKgrBdeuBsxSLHd3G86MQp1OjEH.01.0w056qjbu@gitlab/root/test-flake.git gitlab # [ 178.027116] setup-gitlab-start[2566]: + git push -u origin master gitlab # [ 178.068941] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:14 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 22 gitlab # [ 178.143865] gitaly[1358]: time="2026-07-21T07:46:14.834Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G3ACZGFRB32GYG15HJV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:14.827" grpc.time_ms=6.748 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 178.173246] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:14 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.54.0" 101 gitlab # [ 178.452528] gitaly[1358]: time="2026-07-21T07:46:15.143Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-21T07:46:25.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:15.133" grpc.time_ms=10.348 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 178.480295] gitaly[1358]: time="2026-07-21T07:46:15.171Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=331 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-21T07:46:45.065" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-21T07:46:15.165" grpc.time_ms=5.603 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 178.529719] gitaly[1358]: time="2026-07-21T07:46:15.220Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ duration_ms=248ns method=POST pid=1358 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 178.560682] gitaly[1358]: time="2026-07-21T07:46:15.251Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ duration_ms=25ns method=POST pid=1358 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 178.560850] gitaly[1358]: time="2026-07-21T07:46:15.251Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:14.972" grpc.time_ms=279.66 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 178.604176] gitaly[1358]: time="2026-07-21T07:46:15.295Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:15.294" grpc.time_ms=0.439 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 178.639642] gitaly[1358]: time="2026-07-21T07:46:15.330Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:15.330" grpc.time_ms=0.285 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 178.666796] gitaly[1358]: time="2026-07-21T07:46:15.358Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:15.357" grpc.time_ms=0.221 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 178.722212] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:15.412Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 7e3d3d46e153dc00125adc3d64f351f6b03d321e refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"ddc2e7c5626d65e8a17e67a2","created_at":"2026-07-21T07:46:15.408Z","trace_propagation_headers":{"sentry-trace":"b7717bda02b248f8bdc98fdc6bfb4d2c-90ce885f4266433a","baggage":"sentry-trace_id=b7717bda02b248f8bdc98fdc6bfb4d2c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7G6TEND5VGENS3HTG1YJ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:15.411Z","job_size_bytes":185,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-ddc2e7c5626d65e8a17e67a2: start","job_status":"start","queue_duration_s":0.001117,"scheduling_latency_s":0.001149} gitlab # [ 178.751454] gitaly[1358]: time="2026-07-21T07:46:15.442Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-21T07:46:25.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:15.437" grpc.time_ms=4.834 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 178.757362] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:15.448Z","message":"changes: \"0000000000000000000000000000000000000000 7e3d3d46e153dc00125adc3d64f351f6b03d321e refs/heads/master\\n\""} gitlab # [ 178.777071] gitaly[1358]: time="2026-07-21T07:46:15.467Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1576 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=10 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-21T07:46:25.016" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:15.446" grpc.time_ms=20.423 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 178.783633] gitaly[1358]: time="2026-07-21T07:46:15.474Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ duration_ms=90ns method=POST pid=1358 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 178.784908] gitaly[1358]: time="2026-07-21T07:46:15.475Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:15.384" grpc.time_ms=91.097 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 178.810252] gitaly[1358]: time="2026-07-21T07:46:15.501Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ diskcache=953f009e-40c7-46f9-b249-849602550e78 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:14.920" pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 178.810614] gitaly[1358]: time="2026-07-21T07:46:15.501Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=160 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13153 command.oublock=32 command.real_time_ms=573 command.spawn_token_fork_ms=0 command.system_time_ms=109 command.user_time_ms=51 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:14.920" grpc.time_ms=581.499 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 178.812026] gitaly[1358]: time="2026-07-21T07:46:15.503Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T07:46:25.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:15.496" grpc.time_ms=6.233 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 178.845880] gitlab-workhorse[1289]: time="2026-07-21T07:46:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 178.847908] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:15 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 662 gitlab # [ 178.851406] setup-gitlab-start[2856]: To http://gitlab/root/test-flake.git gitlab # [ 178.852039] setup-gitlab-start[2856]: * [new branch] master -> master gitlab # [ 178.854711] setup-gitlab-start[2856]: branch 'master' set up to track 'origin/master'. gitlab # [ 178.856889] systemd[1]: Finished setup-gitlab.service. gitlab # [ 178.858761] gitaly[1358]: time="2026-07-21T07:46:15.550Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-21T07:46:25.052" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:15.543" grpc.time_ms=6.719 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 178.870879] systemd[1]: Starting nixbot CI service... gitlab # [ 178.889690] gitaly[1358]: time="2026-07-21T07:46:15.580Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-21T07:46:25.083" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=226 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:15.573" grpc.time_ms=7.46 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 178.921067] gitaly[1358]: time="2026-07-21T07:46:15.611Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3942 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:15.611" grpc.time_ms=0.28 pid=1358 span.kind=server system=grpc gitlab # [ 178.947562] gitaly[1358]: time="2026-07-21T07:46:15.638Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3944 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:15.638" grpc.time_ms=0.271 pid=1358 span.kind=server system=grpc gitlab # [ 178.970325] gitaly[1358]: time="2026-07-21T07:46:15.661Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ diskcache=bfb8e301-0eda-44a8-837a-12e280180ba5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-21T07:46:25.103" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:15.593" pid=1358 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 178.970721] gitaly[1358]: time="2026-07-21T07:46:15.661Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=56 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4770 command.oublock=8 command.real_time_ms=62 command.spawn_token_fork_ms=0 command.system_time_ms=42 command.user_time_ms=14 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-21T07:46:25.103" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:15.593" grpc.time_ms=68.118 pid=1358 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 178.983852] gitaly[1358]: time="2026-07-21T07:46:15.675Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-07-21T07:46:25.184" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:15.674" grpc.time_ms=0.13 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 178.993638] gitaly[1358]: time="2026-07-21T07:46:15.684Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=940 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-07-21T07:46:25.188" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:15.678" grpc.time_ms=6.631 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.042098] gitaly[1358]: time="2026-07-21T07:46:15.733Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=342 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-07-21T07:46:45.029" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:15.729" grpc.time_ms=3.875 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.060795] gitaly[1358]: time="2026-07-21T07:46:15.752Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-21T07:46:25.261" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:15.751" grpc.time_ms=0.18 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.065964] gitaly[1358]: time="2026-07-21T07:46:15.757Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=344 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-21T07:46:25.263" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-07-21T07:46:15.753" grpc.time_ms=3.462 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.080423] gitaly[1358]: time="2026-07-21T07:46:15.771Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T07:46:25.276" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:15.766" grpc.time_ms=4.933 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.089313] gitaly[1358]: time="2026-07-21T07:46:15.780Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T07:46:25.284" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:15.774" grpc.time_ms=6.206 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.104742] gitaly[1358]: time="2026-07-21T07:46:15.795Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=967 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-21T07:46:45.086" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:15.787" grpc.time_ms=8.767 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.347195] gitaly[1358]: time="2026-07-21T07:46:16.037Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-21T07:46:26.005" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:16.036" grpc.time_ms=1.554 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.385514] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:16.076Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 7e3d3d46e153dc00125adc3d64f351f6b03d321e refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"ddc2e7c5626d65e8a17e67a2","created_at":"2026-07-21T07:46:15.408Z","trace_propagation_headers":{"sentry-trace":"b7717bda02b248f8bdc98fdc6bfb4d2c-90ce885f4266433a","baggage":"sentry-trace_id=b7717bda02b248f8bdc98fdc6bfb4d2c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7G6TEND5VGENS3HTG1YJ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:15.411Z","job_size_bytes":185,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-ddc2e7c5626d65e8a17e67a2: done: 0.663299 sec","job_status":"done","queue_duration_s":0.001117,"scheduling_latency_s":0.001149,"gitaly_calls":13,"gitaly_duration_s":0.219129,"redis_calls":195,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.032737,"redis_read_bytes":1461,"redis_write_bytes":18936,"redis_cache_calls":14,"redis_cache_duration_s":0.002027,"redis_cache_read_bytes":16,"redis_cache_write_bytes":884,"redis_feature_flag_calls":28,"redis_feature_flag_duration_s":0.007003,"redis_feature_flag_read_bytes":1044,"redis_feature_flag_write_bytes":3487,"redis_queues_calls":8,"redis_queues_duration_s":0.001073,"redis_queues_read_bytes":8,"redis_queues_write_bytes":4934,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003669,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1259,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.010766,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.008199,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.16,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.172,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.35557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000448101,"exclusive_lock_hold_duration_s":0.08364945800000001,"duration_s":0.663299,"completed_at":"2026-07-21T07:46:16.075Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.050792,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 179.391984] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:16.078Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"efa5ba494acb68dfc968ad28","created_at":"2026-07-21T07:46:15.552Z","trace_propagation_headers":{"sentry-trace":"b7717bda02b248f8bdc98fdc6bfb4d2c-b2f8c8f477014710","baggage":"sentry-trace_id=b7717bda02b248f8bdc98fdc6bfb4d2c,sentry-release=Unknown"},"correlation_id":"01KY1T7G6TEND5VGENS3HTG1YJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:15.556Z","job_size_bytes":46,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-efa5ba494acb68dfc968ad28: start","job_status":"start","queue_duration_s":0.521113,"scheduling_latency_s":0.521144} gitlab # [ 179.441778] gitaly[1358]: time="2026-07-21T07:46:16.129Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-07-21T13:46:16.124" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:16.124" grpc.time_ms=4.471 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.457096] gitaly[1358]: time="2026-07-21T07:46:16.148Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-21T07:46:46.038" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:16.139" grpc.time_ms=8.694 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.482785] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:16.173Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"efa5ba494acb68dfc968ad28","created_at":"2026-07-21T07:46:15.552Z","trace_propagation_headers":{"sentry-trace":"b7717bda02b248f8bdc98fdc6bfb4d2c-b2f8c8f477014710","baggage":"sentry-trace_id=b7717bda02b248f8bdc98fdc6bfb4d2c,sentry-release=Unknown"},"correlation_id":"01KY1T7G6TEND5VGENS3HTG1YJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:15.556Z","job_size_bytes":46,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-efa5ba494acb68dfc968ad28: done: 0.095794 sec","job_status":"done","queue_duration_s":0.521113,"scheduling_latency_s":0.521144,"gitaly_calls":2,"gitaly_duration_s":0.017636,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.0058390000000000004,"redis_read_bytes":68,"redis_write_bytes":4886,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001252,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000481,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001371,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002145,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":752,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00059,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000319873,"duration_s":0.095794,"completed_at":"2026-07-21T07:46:16.173Z","load_balancing_strategy":"primary","db_duration_s":0.011757,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 179.505925] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:16.174Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"158436a24090fbb401f08b4a","created_at":"2026-07-21T07:46:15.564Z","trace_propagation_headers":{"sentry-trace":"b7717bda02b248f8bdc98fdc6bfb4d2c-b2f8c8f477014710","baggage":"sentry-trace_id=b7717bda02b248f8bdc98fdc6bfb4d2c,sentry-release=Unknown"},"correlation_id":"01KY1T7G6TEND5VGENS3HTG1YJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:15.570Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-158436a24090fbb401f08b4a: start","job_status":"start","queue_duration_s":0.604799,"scheduling_latency_s":0.604814} gitlab: (finished: waiting for unit setup-gitlab.service, in 180.28 seconds) gitlab # [ 179.559245] gitaly[1358]: time="2026-07-21T07:46:16.250Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ diskcache=0714f607-6925-4e8a-9007-707b651038ad grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-21T13:46:16.188" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:16.189" pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: waiting for unit nixbot.service gitlab # [ 179.568219] gitaly[1358]: time="2026-07-21T07:46:16.250Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1281 command.oublock=0 command.real_time_ms=93 command.spawn_token_fork_ms=38 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-21T13:46:16.188" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:16.189" grpc.time_ms=61.577 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.583107] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:16.274Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"158436a24090fbb401f08b4a","created_at":"2026-07-21T07:46:15.564Z","trace_propagation_headers":{"sentry-trace":"b7717bda02b248f8bdc98fdc6bfb4d2c-b2f8c8f477014710","baggage":"sentry-trace_id=b7717bda02b248f8bdc98fdc6bfb4d2c,sentry-release=Unknown"},"correlation_id":"01KY1T7G6TEND5VGENS3HTG1YJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:15.570Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-158436a24090fbb401f08b4a: done: 0.099279 sec","job_status":"done","queue_duration_s":0.604799,"scheduling_latency_s":0.604814,"gitaly_calls":1,"gitaly_duration_s":0.063211,"redis_calls":15,"redis_duration_s":0.002051,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000252,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000751,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000341,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000707,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.011,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.08468925499999999,"duration_s":0.099279,"completed_at":"2026-07-21T07:46:16.274Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009105,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 179.585060] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:16.275Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"d9af9de6faa32811955f90b1","created_at":"2026-07-21T07:46:15.669Z","trace_propagation_headers":{"sentry-trace":"b7717bda02b248f8bdc98fdc6bfb4d2c-b2f8c8f477014710","baggage":"sentry-trace_id=b7717bda02b248f8bdc98fdc6bfb4d2c,sentry-release=Unknown"},"correlation_id":"01KY1T7G6TEND5VGENS3HTG1YJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:15.672Z","job_size_bytes":23,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-d9af9de6faa32811955f90b1: start","job_status":"start","queue_duration_s":0.602996,"scheduling_latency_s":0.603011} gitlab # [ 179.606702] gitaly[1358]: time="2026-07-21T07:46:16.297Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-07-21T07:46:46.191" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:16.292" grpc.time_ms=5.57 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.653676] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:16.344Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"d9af9de6faa32811955f90b1","created_at":"2026-07-21T07:46:15.669Z","trace_propagation_headers":{"sentry-trace":"b7717bda02b248f8bdc98fdc6bfb4d2c-b2f8c8f477014710","baggage":"sentry-trace_id=b7717bda02b248f8bdc98fdc6bfb4d2c,sentry-release=Unknown"},"correlation_id":"01KY1T7G6TEND5VGENS3HTG1YJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:15.672Z","job_size_bytes":23,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-d9af9de6faa32811955f90b1: done: 0.069445 sec","job_status":"done","queue_duration_s":0.602996,"scheduling_latency_s":0.603011,"gitaly_calls":1,"gitaly_duration_s":0.04628,"redis_calls":16,"redis_duration_s":0.0023510000000000002,"redis_read_bytes":52,"redis_write_bytes":2045,"redis_queues_calls":1,"redis_queues_duration_s":0.000216,"redis_queues_read_bytes":1,"redis_queues_write_bytes":976,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000693,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001031,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000411,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.069445,"completed_at":"2026-07-21T07:46:16.344Z","load_balancing_strategy":"primary","db_duration_s":0.001968,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 179.696612] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:16.387Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"442e30e1305d6fcfe1ca953c","created_at":"2026-07-21T07:46:15.987Z","trace_propagation_headers":{"sentry-trace":"b7717bda02b248f8bdc98fdc6bfb4d2c-b2f8c8f477014710","baggage":"sentry-trace_id=b7717bda02b248f8bdc98fdc6bfb4d2c,sentry-release=Unknown"},"correlation_id":"01KY1T7G6TEND5VGENS3HTG1YJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fe87cc1a86b82bfb032a997ca6024d7b3eff2f4ced4810ad85b04cd11b52d33","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:15.994Z","job_size_bytes":467,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-442e30e1305d6fcfe1ca953c: start","job_status":"start","queue_duration_s":0.392572,"scheduling_latency_s":0.392633} gitlab # [ 179.743634] gitaly[1358]: time="2026-07-21T07:46:16.434Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:46.333" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:16.433" grpc.time_ms=0.656 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.751586] gitaly[1358]: time="2026-07-21T07:46:16.442Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-21T07:46:26.006" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:16.436" grpc.time_ms=5.725 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.827909] gitaly[1358]: time="2026-07-21T07:46:16.519Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-21T07:46:46.417" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:16.517" grpc.time_ms=1.094 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.852411] gitaly[1358]: time="2026-07-21T07:46:16.524Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=264 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-21T07:46:46.421" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:16.521" grpc.time_ms=3.273 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 179.916706] gitaly[1358]: time="2026-07-21T07:46:16.607Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T07:46:26.170" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:16.601" grpc.time_ms=6.692 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 180.052183] nixbot[2951]: {"ts": "2026-07-21T07:46:16.742Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 180.071275] nixbot[2951]: {"ts": "2026-07-21T07:46:16.762Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 180.087289] nixbot[2951]: {"ts": "2026-07-21T07:46:16.778Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 180.090650] nixbot[2951]: {"ts": "2026-07-21T07:46:16.781Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 180.094540] nixbot[2951]: {"ts": "2026-07-21T07:46:16.785Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 180.099513] nixbot[2951]: {"ts": "2026-07-21T07:46:16.790Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 180.103516] nixbot[2951]: {"ts": "2026-07-21T07:46:16.794Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 180.106654] nixbot[2951]: {"ts": "2026-07-21T07:46:16.797Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 180.111666] nixbot[2951]: {"ts": "2026-07-21T07:46:16.802Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 180.114517] nixbot[2951]: {"ts": "2026-07-21T07:46:16.805Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 180.124234] nixbot[2951]: {"ts": "2026-07-21T07:46:16.815Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 180.136633] nixbot[2951]: {"ts": "2026-07-21T07:46:16.827Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 180.140422] nixbot[2951]: {"ts": "2026-07-21T07:46:16.831Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 180.146084] nixbot[2951]: {"ts": "2026-07-21T07:46:16.835Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 180.149122] nixbot[2951]: {"ts": "2026-07-21T07:46:16.840Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 180.152921] nixbot[2951]: {"ts": "2026-07-21T07:46:16.844Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 180.156447] nixbot[2951]: {"ts": "2026-07-21T07:46:16.847Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 180.159781] nixbot[2951]: {"ts": "2026-07-21T07:46:16.851Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 180.166509] nixbot[2951]: {"ts": "2026-07-21T07:46:16.857Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 180.173438] nixbot[2951]: {"ts": "2026-07-21T07:46:16.864Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 180.176639] nixbot[2951]: {"ts": "2026-07-21T07:46:16.867Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 180.182521] nixbot[2951]: {"ts": "2026-07-21T07:46:16.873Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 180.185517] nixbot[2951]: {"ts": "2026-07-21T07:46:16.876Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "effect_deps"} gitlab # [ 180.428268] gitaly[1358]: time="2026-07-21T07:46:17.118Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=965 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-21T07:46:47.002" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:17.102" grpc.time_ms=15.915 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 180.453786] gitaly[1358]: time="2026-07-21T07:46:17.145Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7G6TEND5VGENS3HTG1YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-21T07:46:47.030" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:17.131" grpc.time_ms=13.792 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 180.490245] nixbot[2951]: INFO: Started server process [2951] gitlab # [ 180.490912] nixbot[2951]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 180.493345] nixbot[2951]: INFO: ::ffff:127.0.0.1:33476 - "GET /health HTTP/1.1" 200 OK gitlab # [ 180.498338] systemd[1]: Started nixbot CI service. gitlab # [ 180.499586] systemd[1]: Reached target Multi-User System. gitlab # [ 180.501398] systemd[1]: Startup finished in 1.235s (kernel) + 4.190s (initrd) + 2min 55.075s (userspace) = 3min 501ms. gitlab # [ 180.525994] sidekiq-cluster[2039]: {"severity":"WARN","time":"2026-07-21T07:46:17.216Z","class":"CreatePipelineWorker","correlation_id":"01KY1T7G6TEND5VGENS3HTG1YJ","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"7e3d3d46e153dc00125adc3d64f351f6b03d321e","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"7e3d3d46e153dc00125adc3d64f351f6b03d321e","ref":"refs/heads/master"},"retry":0} gitlab # [ 180.543218] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:17.233Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"442e30e1305d6fcfe1ca953c","created_at":"2026-07-21T07:46:15.987Z","trace_propagation_headers":{"sentry-trace":"b7717bda02b248f8bdc98fdc6bfb4d2c-b2f8c8f477014710","baggage":"sentry-trace_id=b7717bda02b248f8bdc98fdc6bfb4d2c,sentry-release=Unknown"},"correlation_id":"01KY1T7G6TEND5VGENS3HTG1YJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6fe87cc1a86b82bfb032a997ca6024d7b3eff2f4ced4810ad85b04cd11b52d33","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:15.994Z","job_size_bytes":467,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-442e30e1305d6fcfe1ca953c: done: 0.845736 sec","job_status":"done","queue_duration_s":0.392572,"scheduling_latency_s":0.392633,"gitaly_calls":7,"gitaly_duration_s":0.109305,"redis_calls":53,"redis_duration_s":0.015323,"redis_read_bytes":171,"redis_write_bytes":4427,"redis_cache_calls":4,"redis_cache_duration_s":0.000899,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.003719,"redis_feature_flag_read_bytes":18,"redis_feature_flag_write_bytes":2433,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007722,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002242,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000741,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":43,"db_write_count":1,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":43,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":38,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.619101,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00087162,"duration_s":0.845736,"completed_at":"2026-07-21T07:46:17.233Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.065567,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 180.587915] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:17.278Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"0ffc4936215d752d96ca9091","created_at":"2026-07-21T07:46:16.156Z","trace_propagation_headers":{"sentry-trace":"b7717bda02b248f8bdc98fdc6bfb4d2c-fdd28bd47ca84840","baggage":"sentry-trace_id=b7717bda02b248f8bdc98fdc6bfb4d2c,sentry-release=Unknown"},"correlation_id":"01KY1T7G6TEND5VGENS3HTG1YJ","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:16.161Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-0ffc4936215d752d96ca9091: start","job_status":"start","queue_duration_s":1.117453,"scheduling_latency_s":1.117494} gitlab # [ 180.657880] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:17.348Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"0ffc4936215d752d96ca9091","created_at":"2026-07-21T07:46:16.156Z","trace_propagation_headers":{"sentry-trace":"b7717bda02b248f8bdc98fdc6bfb4d2c-fdd28bd47ca84840","baggage":"sentry-trace_id=b7717bda02b248f8bdc98fdc6bfb4d2c,sentry-release=Unknown"},"correlation_id":"01KY1T7G6TEND5VGENS3HTG1YJ","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:16.161Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-0ffc4936215d752d96ca9091: done: 0.069944 sec","job_status":"done","queue_duration_s":1.117453,"scheduling_latency_s":1.117494,"redis_calls":19,"redis_duration_s":0.005091,"redis_read_bytes":21,"redis_write_bytes":4288,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001041,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000729,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2259,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002404,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040803,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.009995683000000002,"duration_s":0.069944,"completed_at":"2026-07-21T07:46:17.348Z","load_balancing_strategy":"primary","db_duration_s":0.006801,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: waiting for unit nixbot.service, in 1.04 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 180.659687] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:17.350Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"27410fb5492148acb9585d18","created_at":"2026-07-21T07:46:17.331Z","trace_propagation_headers":{"sentry-trace":"b7717bda02b248f8bdc98fdc6bfb4d2c-aef83ae2668f41e3","baggage":"sentry-trace_id=b7717bda02b248f8bdc98fdc6bfb4d2c,sentry-release=Unknown"},"correlation_id":"01KY1T7G6TEND5VGENS3HTG1YJ","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:17.339Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-27410fb5492148acb9585d18: start","job_status":"start","queue_duration_s":0.01136,"scheduling_latency_s":0.011385} gitlab # [ 180.726438] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:17.417Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"27410fb5492148acb9585d18","created_at":"2026-07-21T07:46:17.331Z","trace_propagation_headers":{"sentry-trace":"b7717bda02b248f8bdc98fdc6bfb4d2c-aef83ae2668f41e3","baggage":"sentry-trace_id=b7717bda02b248f8bdc98fdc6bfb4d2c,sentry-release=Unknown"},"correlation_id":"01KY1T7G6TEND5VGENS3HTG1YJ","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:17.339Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-27410fb5492148acb9585d18: done: 0.066724 sec","job_status":"done","queue_duration_s":0.01136,"scheduling_latency_s":0.011385,"redis_calls":11,"redis_duration_s":0.002865,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000422,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001783,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00066,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000265117,"duration_s":0.066724,"completed_at":"2026-07-21T07:46:17.417Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020388,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 180.754211] gitaly[1358]: time="2026-07-21T07:46:17.445Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=965 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7JEKJKB03ZPWFCXG3QMS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-21T07:46:47.035" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:17.435" grpc.time_ms=9.389 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 180.778103] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:17 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 296 gitlab # [ 180.778947] nixbot[2951]: {"ts": "2026-07-21T07:46:17.469Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 180.786378] nixbot[2951]: INFO: ::ffff:127.0.0.1:33478 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.08 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 180.790460] nixbot[2951]: {"ts": "2026-07-21T07:46:17.481Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 180.840718] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 43 gitlab # [ 180.842049] nixbot[2951]: {"ts": "2026-07-21T07:46:17.533Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 180.843812] nixbot[2951]: {"ts": "2026-07-21T07:46:17.535Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 180.876069] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:17 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.20.0" 66 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.09 seconds) gitlab # [ 180.929497] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 868 "" "python-httpx/0.28.1" 82 gitlab # [ 180.930333] nixbot[2951]: {"ts": "2026-07-21T07:46:17.621Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 181.975738] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:18 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "curl/8.20.0" 60 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds) (finished: subtest: project discovered and webhook registered, in 1.19 seconds) subtest: push delivers the webhook gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 182.068457] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:18 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.54.0" 33 gitlab # [ 182.139719] gitaly[1358]: time="2026-07-21T07:46:18.830Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=644 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M0CHCQPHS9XAVDMDYRQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:18.820" grpc.time_ms=10.011 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 182.141267] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:18 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 199 "" "git/2.54.0" 67 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 274.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 182.302623] gitaly[1358]: time="2026-07-21T07:46:18.993Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-21T07:46:28.257" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=253 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:18.987" grpc.time_ms=5.809 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 182.424751] gitaly[1358]: time="2026-07-21T07:46:19.115Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-07-21T07:46:49.006" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-07-21T07:46:19.107" grpc.time_ms=8.777 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 182.470331] gitaly[1358]: time="2026-07-21T07:46:19.161Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q duration_ms=212ns method=POST pid=1358 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 182.498656] gitaly[1358]: time="2026-07-21T07:46:19.189Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q duration_ms=21ns method=POST pid=1358 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 182.499497] gitaly[1358]: time="2026-07-21T07:46:19.190Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5322 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:18.948" grpc.time_ms=242.546 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 182.549744] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:19 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4876 "" "python-httpx/0.28.1" 1614 gitlab # [ 182.553457] gitaly[1358]: time="2026-07-21T07:46:19.244Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4792 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:19.244" grpc.time_ms=0.427 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 182.566818] nixbot[2951]: {"ts": "2026-07-21T07:46:19.257Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 182.592593] gitaly[1358]: time="2026-07-21T07:46:19.283Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4791 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:19.283" grpc.time_ms=0.347 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 182.625045] gitaly[1358]: time="2026-07-21T07:46:19.314Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4793 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:19.314" grpc.time_ms=0.638 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 182.650741] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:19 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 831 "" "python-httpx/0.28.1" 72 gitlab # [ 182.652101] nixbot[2951]: {"ts": "2026-07-21T07:46:19.343Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 182.691549] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:19.380Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","7e3d3d46e153dc00125adc3d64f351f6b03d321e 7499edfcc425b7c57426f5a658af2ef46637209f refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"70c5d84096deaf5852cfa6f0","created_at":"2026-07-21T07:46:19.376Z","trace_propagation_headers":{"sentry-trace":"f282444135154e80a2b725721b038e2c-99855f194d054641","baggage":"sentry-trace_id=f282444135154e80a2b725721b038e2c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7M3ACBT2MQMZMCZKM24Q","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:19.378Z","job_size_bytes":185,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-70c5d84096deaf5852cfa6f0: start","job_status":"start","queue_duration_s":0.002295,"scheduling_latency_s":0.002323} gitlab # [ 182.700874] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:19.392Z","message":"changes: \"7e3d3d46e153dc00125adc3d64f351f6b03d321e 7499edfcc425b7c57426f5a658af2ef46637209f refs/heads/master\\n\""} gitlab # [ 182.708039] gitaly[1358]: time="2026-07-21T07:46:19.398Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q duration_ms=50ns method=POST pid=1358 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 182.709961] gitaly[1358]: time="2026-07-21T07:46:19.399Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4816 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:19.347" grpc.time_ms=51.887 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 182.728855] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:19 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 868 "" "python-httpx/0.28.1" 73 gitlab # [ 182.729463] nixbot[2951]: {"ts": "2026-07-21T07:46:19.420Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 182.739521] gitaly[1358]: time="2026-07-21T07:46:19.430Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=15 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T07:46:29.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:19.409" grpc.time_ms=21.659 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 182.759161] gitaly[1358]: time="2026-07-21T07:46:19.450Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-07-21T07:46:29.013" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:19.444" grpc.time_ms=6.12 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 182.773226] gitaly[1358]: time="2026-07-21T07:46:19.463Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q diskcache=44881102-cda2-4dcd-90b2-9f82b95503be grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:18.901" pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # To http://gitlab/root/test-flake.git gitlab # 7e3d3d4..7499edf master -> master gitlab # [ 182.783496] gitlab-workhorse[1289]: time="2026-07-21T07:46:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 182.786212] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:19 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.54.0" 605 gitlab # [ 182.788159] gitaly[1358]: time="2026-07-21T07:46:19.463Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7MN0SX4D57W19HS8160P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-07-21T07:46:49.055" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=255 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:19.455" grpc.time_ms=8.023 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 182.802065] gitaly[1358]: time="2026-07-21T07:46:19.463Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=149 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=897024 command.minflt=13491 command.oublock=32 command.real_time_ms=526 command.spawn_token_fork_ms=2 command.system_time_ms=110 command.user_time_ms=39 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:18.901" grpc.time_ms=561.778 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 182.802589] gitaly[1358]: time="2026-07-21T07:46:19.477Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7MN0SX4D57W19HS8160P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:49.075" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:19.476" grpc.time_ms=0.9 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 182.803482] gitaly[1358]: time="2026-07-21T07:46:19.486Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=4 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-07-21T07:46:29.046" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:19.477" grpc.time_ms=9.4 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 182.805103] gitaly[1358]: time="2026-07-21T07:46:19.496Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7MN0SX4D57W19HS8160P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-07-21T07:46:29.039" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:19.479" grpc.time_ms=16.382 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 182.830476] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:19 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 771 "" "python-httpx/0.28.1" 97 gitlab # [ 182.831626] nixbot[2951]: {"ts": "2026-07-21T07:46:19.522Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.88 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab # [ 182.864570] gitaly[1358]: time="2026-07-21T07:46:19.555Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-07-21T07:46:49.447" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:19.548" grpc.time_ms=7.623 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7499edfcc425b7c57426f5a658af2ef46637209f' gitlab # [ 182.886955] gitaly[1358]: time="2026-07-21T07:46:19.578Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=343 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-07-21T07:46:29.144" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-07-21T07:46:19.575" grpc.time_ms=3.075 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 182.896674] nixbot[2951]: INFO: ::ffff:127.0.0.1:33486 - "GET /api/repos/gitlab/root/test-flake/builds?commit=7499edfcc425b7c57426f5a658af2ef46637209f HTTP/1.1" 200 OK gitlab # [ 182.898893] gitaly[1358]: time="2026-07-21T07:46:19.590Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T07:46:29.153" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:19.584" grpc.time_ms=6.043 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7499edfcc425b7c57426f5a658af2ef46637209f', in 0.03 seconds) gitlab # [ 182.923276] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:19 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 88 gitlab # [ 182.925202] nixbot[2951]: {"ts": "2026-07-21T07:46:19.615Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 182.927098] nixbot[2951]: {"ts": "2026-07-21T07:46:19.618Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 182.990980] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:19 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 38 gitlab # [ 182.991508] gitaly[1358]: time="2026-07-21T07:46:19.680Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7MVTSY26A6V1AQHFV6PQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:19.674" grpc.time_ms=5.944 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 183.033075] gitaly[1358]: time="2026-07-21T07:46:19.723Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-07-21T07:46:29.002" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:19.722" grpc.time_ms=1.172 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 183.055444] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:19.746Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","7e3d3d46e153dc00125adc3d64f351f6b03d321e 7499edfcc425b7c57426f5a658af2ef46637209f refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"70c5d84096deaf5852cfa6f0","created_at":"2026-07-21T07:46:19.376Z","trace_propagation_headers":{"sentry-trace":"f282444135154e80a2b725721b038e2c-99855f194d054641","baggage":"sentry-trace_id=f282444135154e80a2b725721b038e2c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7M3ACBT2MQMZMCZKM24Q","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-07-21T07:46:19.378Z","job_size_bytes":185,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-70c5d84096deaf5852cfa6f0: done: 0.365728 sec","job_status":"done","queue_duration_s":0.002295,"scheduling_latency_s":0.002323,"gitaly_calls":7,"gitaly_duration_s":0.158297,"redis_calls":108,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.014715,"redis_read_bytes":349,"redis_write_bytes":12824,"redis_cache_calls":5,"redis_cache_duration_s":0.001081,"redis_cache_read_bytes":13,"redis_cache_write_bytes":347,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":323,"redis_queues_calls":8,"redis_queues_duration_s":0.000786,"redis_queues_read_bytes":8,"redis_queues_write_bytes":6770,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.002117,"redis_queues_metadata_read_bytes":11,"redis_queues_metadata_write_bytes":1182,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.005338,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.005112,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.02,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.02,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.169816,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000674387,"exclusive_lock_hold_duration_s":0.029301769,"duration_s":0.365728,"completed_at":"2026-07-21T07:46:19.746Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.050085,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 183.109872] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:19.800Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"47e86350454e55fd4cf211b6","created_at":"2026-07-21T07:46:19.466Z","trace_propagation_headers":{"sentry-trace":"f282444135154e80a2b725721b038e2c-da4366fff17a4be7","baggage":"sentry-trace_id=f282444135154e80a2b725721b038e2c,sentry-release=Unknown"},"correlation_id":"01KY1T7M3ACBT2MQMZMCZKM24Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:19.468Z","job_size_bytes":46,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-47e86350454e55fd4cf211b6: start","job_status":"start","queue_duration_s":0.331918,"scheduling_latency_s":0.331944} gitlab # [ 183.124493] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:19.815Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"47e86350454e55fd4cf211b6","created_at":"2026-07-21T07:46:19.466Z","trace_propagation_headers":{"sentry-trace":"f282444135154e80a2b725721b038e2c-da4366fff17a4be7","baggage":"sentry-trace_id=f282444135154e80a2b725721b038e2c,sentry-release=Unknown"},"correlation_id":"01KY1T7M3ACBT2MQMZMCZKM24Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:19.468Z","job_size_bytes":46,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-47e86350454e55fd4cf211b6: done: 0.01511 sec","job_status":"done","queue_duration_s":0.331918,"scheduling_latency_s":0.331944,"redis_calls":9,"redis_duration_s":0.001742,"redis_read_bytes":23,"redis_write_bytes":782,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000952,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":1,"redis_repository_cache_duration_s":0.000217,"redis_repository_cache_read_bytes":18,"redis_repository_cache_write_bytes":41,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000573,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":304,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01511,"completed_at":"2026-07-21T07:46:19.815Z","load_balancing_strategy":"primary","db_duration_s":0.001743,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 183.127049] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:19.818Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"23ccd4a79eca40243f3ed067","created_at":"2026-07-21T07:46:19.473Z","trace_propagation_headers":{"sentry-trace":"f282444135154e80a2b725721b038e2c-da4366fff17a4be7","baggage":"sentry-trace_id=f282444135154e80a2b725721b038e2c,sentry-release=Unknown"},"correlation_id":"01KY1T7M3ACBT2MQMZMCZKM24Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:19.474Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-23ccd4a79eca40243f3ed067: start","job_status":"start","queue_duration_s":0.343505,"scheduling_latency_s":0.343519} gitlab # [ 183.210370] gitaly[1358]: time="2026-07-21T07:46:19.901Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q diskcache=1892f2e2-bfd7-4bd0-8309-7ec5076ddd99 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-21T13:46:19.835" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:19.836" pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 183.211053] gitaly[1358]: time="2026-07-21T07:46:19.902Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1647 command.oublock=0 command.real_time_ms=105 command.spawn_token_fork_ms=41 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-07-21T13:46:19.835" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:19.836" grpc.time_ms=65.036 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 183.222072] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:19.913Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"23ccd4a79eca40243f3ed067","created_at":"2026-07-21T07:46:19.473Z","trace_propagation_headers":{"sentry-trace":"f282444135154e80a2b725721b038e2c-da4366fff17a4be7","baggage":"sentry-trace_id=f282444135154e80a2b725721b038e2c,sentry-release=Unknown"},"correlation_id":"01KY1T7M3ACBT2MQMZMCZKM24Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:19.474Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-23ccd4a79eca40243f3ed067: done: 0.094942 sec","job_status":"done","queue_duration_s":0.343505,"scheduling_latency_s":0.343519,"gitaly_calls":1,"gitaly_duration_s":0.06742,"redis_calls":12,"redis_duration_s":0.004582,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002542,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000351,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001689,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.076292657,"duration_s":0.094942,"completed_at":"2026-07-21T07:46:19.913Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005318,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 183.236886] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:19.918Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"fe8cfbfe4bcc2bae886de39f","created_at":"2026-07-21T07:46:19.634Z","trace_propagation_headers":{"sentry-trace":"f282444135154e80a2b725721b038e2c-da4366fff17a4be7","baggage":"sentry-trace_id=f282444135154e80a2b725721b038e2c,sentry-release=Unknown"},"correlation_id":"01KY1T7M3ACBT2MQMZMCZKM24Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3419bb60f588cbbb06a76d42f786837ee46c3e2cf574dfbf1784ab7ea6dfaf52","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:19.636Z","job_size_bytes":467,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-fe8cfbfe4bcc2bae886de39f: start","job_status":"start","queue_duration_s":0.282556,"scheduling_latency_s":0.28257} gitlab # [ 183.282660] gitaly[1358]: time="2026-07-21T07:46:19.937Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:49.836" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:19.936" grpc.time_ms=0.559 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 183.283626] gitaly[1358]: time="2026-07-21T07:46:19.942Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-21T07:46:29.218" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:19.938" grpc.time_ms=4.572 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 183.284547] gitaly[1358]: time="2026-07-21T07:46:19.960Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-21T07:46:49.859" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:19.959" grpc.time_ms=0.458 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 183.293605] gitaly[1358]: time="2026-07-21T07:46:19.963Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=101 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-07-21T07:46:49.862" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:19.962" grpc.time_ms=1.501 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 183.348249] gitaly[1358]: time="2026-07-21T07:46:20.002Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T07:46:29.277" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:19.997" grpc.time_ms=4.984 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 183.386345] gitaly[1358]: time="2026-07-21T07:46:20.077Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7MXH4WXDPPWNBHTBWAC1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:20.070" pid=1358 remote_ip="2001:db8:1::1" request_sha=083c43590fffa527704b32b8836424e8ed055017 response_bytes=149 span.kind=server system=grpc gitlab # [ 183.387156] gitaly[1358]: time="2026-07-21T07:46:20.077Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7MXH4WXDPPWNBHTBWAC1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:20.070" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 183.410495] gitaly[1358]: time="2026-07-21T07:46:20.101Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=639 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7MXH4WXDPPWNBHTBWAC1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:20.070" grpc.time_ms=30.618 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 183.425609] gitlab-workhorse[1289]: time="2026-07-21T07:46:20Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=149 gitlab # [ 183.427742] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:20 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 149 "" "git/2.54.0" 419 gitlab # [ 183.482319] gitaly[1358]: time="2026-07-21T07:46:20.173Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NAR403PBJ2ZP496HHBV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:20.161" pid=1358 remote_ip="2001:db8:1::1" request_sha=7bcf64dcc9d241f1a3e98cb378ffb20a2ae204a4 response_bytes=423 span.kind=server system=grpc gitlab # [ 183.482770] gitaly[1358]: time="2026-07-21T07:46:20.173Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1195 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NAR403PBJ2ZP496HHBV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:20.161" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 183.507588] gitaly[1358]: time="2026-07-21T07:46:20.198Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1195 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NAR403PBJ2ZP496HHBV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:20.161" grpc.time_ms=36.82 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=2 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 183.508489] gitlab-workhorse[1289]: time="2026-07-21T07:46:20Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=423 gitlab # [ 183.509083] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:20 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 423 "" "git/2.54.0" 78 gitlab # [ 183.589978] gitaly[1358]: time="2026-07-21T07:46:20.281Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7NEN8WK6Q260Z8NAP4W0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:20.274" grpc.time_ms=6.966 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 183.608054] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:20 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 53 gitlab # [ 183.661932] gitaly[1358]: time="2026-07-21T07:46:20.353Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NGDPH4GD96PD6C9N8YR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:20.344" pid=1358 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc gitlab # [ 183.662513] gitaly[1358]: time="2026-07-21T07:46:20.353Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NGDPH4GD96PD6C9N8YR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:20.344" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 183.664273] gitaly[1358]: time="2026-07-21T07:46:20.355Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NGDPH4GD96PD6C9N8YR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:20.344" grpc.time_ms=11.324 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 183.700066] gitlab-workhorse[1289]: time="2026-07-21T07:46:20Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 183.700621] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:20 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 89 gitlab # [ 183.763683] gitaly[1358]: time="2026-07-21T07:46:20.454Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7NM2SQWN8YS8K4AQKA0S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:20.447" grpc.time_ms=7.121 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 183.777023] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:20 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 49 gitlab # [ 183.826969] gitaly[1358]: time="2026-07-21T07:46:20.516Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NNQM2WP00FZBGV5WXGZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:20.502" pid=1358 remote_ip="2001:db8:1::1" request_sha=f1e6bb59289c4d43fc3699ed3fc4760ee01ef0e9 response_bytes=221 span.kind=server system=grpc gitlab # [ 183.827237] gitlab-workhorse[1289]: time="2026-07-21T07:46:20Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=221 gitlab # [ 183.828102] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:20 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 221 "" "git/2.54.0" 46 gitlab # [ 183.828750] gitaly[1358]: time="2026-07-21T07:46:20.516Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1165 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NNQM2WP00FZBGV5WXGZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:20.502" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 183.830028] gitaly[1358]: time="2026-07-21T07:46:20.517Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1165 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NNQM2WP00FZBGV5WXGZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:20.502" grpc.time_ms=14.486 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=1 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 183.840151] gitaly[1358]: time="2026-07-21T07:46:20.531Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=964 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-07-21T07:46:50.422" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:20.522" grpc.time_ms=8.522 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 183.890624] gitaly[1358]: time="2026-07-21T07:46:20.581Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7M3ACBT2MQMZMCZKM24Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-07-21T07:46:50.473" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:20.574" grpc.time_ms=7.584 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 183.899801] sidekiq-cluster[2039]: {"severity":"WARN","time":"2026-07-21T07:46:20.590Z","class":"CreatePipelineWorker","correlation_id":"01KY1T7M3ACBT2MQMZMCZKM24Q","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"7e3d3d46e153dc00125adc3d64f351f6b03d321e","after":"7499edfcc425b7c57426f5a658af2ef46637209f","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"7499edfcc425b7c57426f5a658af2ef46637209f","ref":"refs/heads/master"},"retry":0} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7499edfcc425b7c57426f5a658af2ef46637209f' gitlab # [ 183.906615] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:20.597Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"fe8cfbfe4bcc2bae886de39f","created_at":"2026-07-21T07:46:19.634Z","trace_propagation_headers":{"sentry-trace":"f282444135154e80a2b725721b038e2c-da4366fff17a4be7","baggage":"sentry-trace_id=f282444135154e80a2b725721b038e2c,sentry-release=Unknown"},"correlation_id":"01KY1T7M3ACBT2MQMZMCZKM24Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:3419bb60f588cbbb06a76d42f786837ee46c3e2cf574dfbf1784ab7ea6dfaf52","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:19.636Z","job_size_bytes":467,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-fe8cfbfe4bcc2bae886de39f: done: 0.678849 sec","job_status":"done","queue_duration_s":0.282556,"scheduling_latency_s":0.28257,"gitaly_calls":7,"gitaly_duration_s":0.082197,"redis_calls":35,"redis_duration_s":0.009151,"redis_read_bytes":188,"redis_write_bytes":2122,"redis_cache_calls":3,"redis_cache_duration_s":0.000525,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000329,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":283,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002558,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.005011,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000728,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":32,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":32,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.076,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.074,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.353666,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.678849,"completed_at":"2026-07-21T07:46:20.597Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.075882,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 183.913667] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:20.598Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"e8714d5a-f54a-4a3b-adbc-fe24eccbbbf7\"}"],"class":"WebHookWorker","jid":"8a016d645fbfbc5fb250420d","created_at":"2026-07-21T07:46:19.711Z","trace_propagation_headers":{"sentry-trace":"f282444135154e80a2b725721b038e2c-da4366fff17a4be7","baggage":"sentry-trace_id=f282444135154e80a2b725721b038e2c,sentry-release=Unknown"},"correlation_id":"01KY1T7M3ACBT2MQMZMCZKM24Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4d6216144867f008235bfcf085c943b7f455d1abc36c0124de0b3b2229238bdb","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:19.715Z","job_size_bytes":1749,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-8a016d645fbfbc5fb250420d: start","job_status":"start","queue_duration_s":0.882804,"scheduling_latency_s":0.882821} gitlab # [ 183.982403] nixbot[2951]: INFO: ::ffff:127.0.0.1:38722 - "GET /api/repos/gitlab/root/test-flake/builds?commit=7499edfcc425b7c57426f5a658af2ef46637209f HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=7499edfcc425b7c57426f5a658af2ef46637209f', in 0.07 seconds) (finished: subtest: push delivers the webhook, in 2.00 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses gitlab # [ 184.015813] nixbot[2951]: INFO: ::ffff:127.0.0.1:38724 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 184.026603] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:20.717Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"e8714d5a-f54a-4a3b-adbc-fe24eccbbbf7\"}"],"class":"WebHookWorker","jid":"8a016d645fbfbc5fb250420d","created_at":"2026-07-21T07:46:19.711Z","trace_propagation_headers":{"sentry-trace":"f282444135154e80a2b725721b038e2c-da4366fff17a4be7","baggage":"sentry-trace_id=f282444135154e80a2b725721b038e2c,sentry-release=Unknown"},"correlation_id":"01KY1T7M3ACBT2MQMZMCZKM24Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4d6216144867f008235bfcf085c943b7f455d1abc36c0124de0b3b2229238bdb","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:19.715Z","job_size_bytes":1749,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-8a016d645fbfbc5fb250420d: done: 0.118795 sec","job_status":"done","queue_duration_s":0.882804,"scheduling_latency_s":0.882821,"redis_calls":32,"redis_duration_s":0.00409,"redis_read_bytes":171,"redis_write_bytes":6966,"redis_feature_flag_calls":17,"redis_feature_flag_duration_s":0.002171,"redis_feature_flag_read_bytes":144,"redis_feature_flag_write_bytes":2306,"redis_queues_calls":2,"redis_queues_duration_s":0.000203,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3644,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00103,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000686,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":10,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.006137372000011965,"cpu_s":0.059617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000229358,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","duration_s":0.118795,"completed_at":"2026-07-21T07:46:20.717Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007639,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 184.047881] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:20.719Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","8a016d645fbfbc5fb250420d"],"class":"WebHooks::LogExecutionWorker","jid":"0fca7742f569c225bbb44a3e","created_at":"2026-07-21T07:46:20.708Z","trace_propagation_headers":{"sentry-trace":"f282444135154e80a2b725721b038e2c-7b5a86f1ed214fb5","baggage":"sentry-trace_id=f282444135154e80a2b725721b038e2c,sentry-release=Unknown"},"correlation_id":"01KY1T7M3ACBT2MQMZMCZKM24Q","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b821cad7e539069753e597dd7626eef168680d9641c06a02cc34878554e5bbe3","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:20.712Z","job_size_bytes":2507,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-0fca7742f569c225bbb44a3e: start","job_status":"start","queue_duration_s":0.006762,"scheduling_latency_s":0.006778} gitlab # [ 184.048722] gitaly[1358]: time="2026-07-21T07:46:20.719Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NVYK15ZY9R6V8BF6YW4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:50.618" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:20.718" grpc.time_ms=0.396 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.057646] gitaly[1358]: time="2026-07-21T07:46:20.730Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NWM4YZE1F5480N8KDNQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:50.329" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:20.729" grpc.time_ms=0.689 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.072878] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses HTTP/1.1" 200 2 "" "curl/8.20.0" 71 gitlab # [ 184.083351] gitaly[1358]: time="2026-07-21T07:46:20.770Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=651 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=17 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7NVYK15ZY9R6V8BF6YW4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-07-21T07:46:50.648" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:20.748" grpc.time_ms=21.187 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses, in 0.14 seconds) gitlab # [ 184.125744] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:20.816Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","8a016d645fbfbc5fb250420d"],"class":"WebHooks::LogExecutionWorker","jid":"0fca7742f569c225bbb44a3e","created_at":"2026-07-21T07:46:20.708Z","trace_propagation_headers":{"sentry-trace":"f282444135154e80a2b725721b038e2c-7b5a86f1ed214fb5","baggage":"sentry-trace_id=f282444135154e80a2b725721b038e2c,sentry-release=Unknown"},"correlation_id":"01KY1T7M3ACBT2MQMZMCZKM24Q","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b821cad7e539069753e597dd7626eef168680d9641c06a02cc34878554e5bbe3","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:20.712Z","job_size_bytes":2507,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-0fca7742f569c225bbb44a3e: done: 0.097225 sec","job_status":"done","queue_duration_s":0.006762,"scheduling_latency_s":0.006778,"redis_calls":10,"redis_duration_s":0.003185,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001219,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001688,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000278,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.024,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.024,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.077405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000310374,"duration_s":0.097225,"completed_at":"2026-07-21T07:46:20.816Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010352,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 184.126650] gitaly[1358]: time="2026-07-21T07:46:20.817Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=20 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7NVYK15ZY9R6V8BF6YW4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T07:46:30.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:20.792" grpc.time_ms=25.14 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.230615] gitaly[1358]: time="2026-07-21T07:46:20.921Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NVYK15ZY9R6V8BF6YW4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-07-21T07:46:30.130" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:20.920" grpc.time_ms=0.358 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.258079] gitaly[1358]: time="2026-07-21T07:46:20.949Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NVYK15ZY9R6V8BF6YW4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-07-21T07:46:30.149" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:20.940" grpc.time_ms=8.599 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.319633] gitaly[1358]: time="2026-07-21T07:46:21.010Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7NVYK15ZY9R6V8BF6YW4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4159 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:21.010" grpc.time_ms=0.529 pid=1358 span.kind=server system=grpc gitlab # [ 184.368614] gitaly[1358]: time="2026-07-21T07:46:21.059Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7NVYK15ZY9R6V8BF6YW4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4161 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:21.059" grpc.time_ms=0.632 pid=1358 span.kind=server system=grpc gitlab # [ 184.393872] gitaly[1358]: time="2026-07-21T07:46:21.085Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NVYK15ZY9R6V8BF6YW4 diskcache=be17ee90-3de3-4f60-88ff-9c1a2e115020 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-21T07:46:30.172" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:20.962" pid=1358 ref=refs/keep-around/7499edfcc425b7c57426f5a658af2ef46637209f remote_ip="2001:db8:1::1" revision=7499edfcc425b7c57426f5a658af2ef46637209f span.kind=server system=grpc username=root gitlab # [ 184.394296] gitaly[1358]: time="2026-07-21T07:46:21.085Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=115 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5436 command.oublock=8 command.real_time_ms=112 command.spawn_token_fork_ms=0 command.system_time_ms=77 command.user_time_ms=38 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NVYK15ZY9R6V8BF6YW4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-07-21T07:46:30.172" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-07-21T07:46:20.962" grpc.time_ms=122.506 pid=1358 ref=refs/keep-around/7499edfcc425b7c57426f5a658af2ef46637209f remote_ip="2001:db8:1::1" revision=7499edfcc425b7c57426f5a658af2ef46637209f span.kind=server system=grpc username=root gitlab # [ 184.399305] gitaly[1358]: time="2026-07-21T07:46:21.090Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NVYK15ZY9R6V8BF6YW4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:51.089" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:21.089" grpc.time_ms=0.509 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.415981] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:21.105Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"45161ee29b6fc8d14cbfabc3","created_at":"2026-07-21T07:46:21.097Z","trace_propagation_headers":{"sentry-trace":"8cf1cd960e4f4812848b9de885176b9c-71606e40f0af40e1","baggage":"sentry-trace_id=8cf1cd960e4f4812848b9de885176b9c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7NVYK15ZY9R6V8BF6YW4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:21.103Z","job_size_bytes":65,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-45161ee29b6fc8d14cbfabc3: start","job_status":"start","queue_duration_s":0.001652,"scheduling_latency_s":0.001699} gitlab # [ 184.473137] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:21.164Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"45161ee29b6fc8d14cbfabc3","created_at":"2026-07-21T07:46:21.097Z","trace_propagation_headers":{"sentry-trace":"8cf1cd960e4f4812848b9de885176b9c-71606e40f0af40e1","baggage":"sentry-trace_id=8cf1cd960e4f4812848b9de885176b9c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7NVYK15ZY9R6V8BF6YW4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:21.103Z","job_size_bytes":65,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-45161ee29b6fc8d14cbfabc3: done: 0.058618 sec","job_status":"done","queue_duration_s":0.001652,"scheduling_latency_s":0.001699,"redis_calls":19,"redis_duration_s":0.004528,"redis_read_bytes":145,"redis_write_bytes":2300,"redis_feature_flag_calls":11,"redis_feature_flag_duration_s":0.002951,"redis_feature_flag_read_bytes":138,"redis_feature_flag_write_bytes":1482,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001176,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000401,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000214273,"duration_s":0.058618,"completed_at":"2026-07-21T07:46:21.164Z","load_balancing_strategy":"primary","db_duration_s":0.011808,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 184.514365] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:21.205Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"a2fca9d9b52c3db5f17f291f","created_at":"2026-07-21T07:46:21.104Z","trace_propagation_headers":{"sentry-trace":"8cf1cd960e4f4812848b9de885176b9c-71606e40f0af40e1","baggage":"sentry-trace_id=8cf1cd960e4f4812848b9de885176b9c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7NVYK15ZY9R6V8BF6YW4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:21.109Z","job_size_bytes":65,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-a2fca9d9b52c3db5f17f291f: start","job_status":"start","queue_duration_s":0.095957,"scheduling_latency_s":0.095995} gitlab # [ 184.546663] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:21 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7499edfcc425b7c57426f5a658af2ef46637209f HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 567 gitlab # [ 184.547560] nixbot[2951]: {"ts": "2026-07-21T07:46:21.238Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7499edfcc425b7c57426f5a658af2ef46637209f \"HTTP/1.1 201 Created\""} gitlab # [ 184.553392] nixbot[2951]: {"ts": "2026-07-21T07:46:21.244Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-7499edfcc425-86e20123", "argv0": "bwrap"} gitlab # [ 184.581606] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:21.272Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"a2fca9d9b52c3db5f17f291f","created_at":"2026-07-21T07:46:21.104Z","trace_propagation_headers":{"sentry-trace":"8cf1cd960e4f4812848b9de885176b9c-71606e40f0af40e1","baggage":"sentry-trace_id=8cf1cd960e4f4812848b9de885176b9c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7NVYK15ZY9R6V8BF6YW4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:21.109Z","job_size_bytes":65,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-a2fca9d9b52c3db5f17f291f: done: 0.067273 sec","job_status":"done","queue_duration_s":0.095957,"scheduling_latency_s":0.095995,"redis_calls":15,"redis_duration_s":0.003458,"redis_read_bytes":142,"redis_write_bytes":1431,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000864,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":431,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001609,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000564318,"duration_s":0.067273,"completed_at":"2026-07-21T07:46:21.272Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010808,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 184.622433] systemd[1]: Started Nix Daemon. gitlab # [ 184.623968] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:21.313Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a324f1e6c0182dcf50444460","created_at":"2026-07-21T07:46:21.223Z","trace_propagation_headers":{"sentry-trace":"8cf1cd960e4f4812848b9de885176b9c-71606e40f0af40e1","baggage":"sentry-trace_id=8cf1cd960e4f4812848b9de885176b9c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7NVYK15ZY9R6V8BF6YW4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:21.226Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a324f1e6c0182dcf50444460: start","job_status":"start","queue_duration_s":0.086796,"scheduling_latency_s":0.086831} gitlab # [ 184.690367] nix-daemon[3271]: accepted connection from pid 3269, user nixbot gitlab # [ 184.691699] nix-daemon[3271]: accepted connection from pid 3270, user nixbot gitlab # [ 184.698181] gitaly[1358]: time="2026-07-21T07:46:21.389Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7NVYK15ZY9R6V8BF6YW4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-21T07:46:51.288" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:21.388" grpc.time_ms=0.815 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.722937] gitaly[1358]: time="2026-07-21T07:46:21.414Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NVYK15ZY9R6V8BF6YW4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:51.313" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:21.413" grpc.time_ms=0.504 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.754466] gitaly[1358]: time="2026-07-21T07:46:21.445Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7NVYK15ZY9R6V8BF6YW4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:51.344" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:21.444" grpc.time_ms=0.695 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 184.763403] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:21.454Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"a324f1e6c0182dcf50444460","created_at":"2026-07-21T07:46:21.223Z","trace_propagation_headers":{"sentry-trace":"8cf1cd960e4f4812848b9de885176b9c-71606e40f0af40e1","baggage":"sentry-trace_id=8cf1cd960e4f4812848b9de885176b9c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7NVYK15ZY9R6V8BF6YW4","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:21.226Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a324f1e6c0182dcf50444460: done: 0.140672 sec","job_status":"done","queue_duration_s":0.086796,"scheduling_latency_s":0.086831,"gitaly_calls":3,"gitaly_duration_s":0.019695,"redis_calls":53,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007617,"redis_read_bytes":235,"redis_write_bytes":7159,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000681,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001657,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_feature_flag_calls":7,"redis_feature_flag_duration_s":0.0014,"redis_feature_flag_read_bytes":134,"redis_feature_flag_write_bytes":992,"redis_queues_calls":2,"redis_queues_duration_s":0.000182,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.001568,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":840,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000257,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.00029,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":34,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":34,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":22,"db_ci_count":12,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084681,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.109191812,"duration_s":0.140672,"completed_at":"2026-07-21T07:46:21.454Z","load_balancing_strategy":"primary","db_duration_s":0.028672,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 184.774163] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:21.455Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8e26eab49b3e8b165a0f3187","created_at":"2026-07-21T07:46:21.373Z","trace_propagation_headers":{"sentry-trace":"8cf1cd960e4f4812848b9de885176b9c-954da9f4a497497e","baggage":"sentry-trace_id=8cf1cd960e4f4812848b9de885176b9c,sentry-release=Unknown"},"correlation_id":"01KY1T7NVYK15ZY9R6V8BF6YW4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:21.376Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8e26eab49b3e8b165a0f3187: start","job_status":"start","queue_duration_s":0.078779,"scheduling_latency_s":0.078793} gitlab # [ 184.823951] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:21.478Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8e26eab49b3e8b165a0f3187","created_at":"2026-07-21T07:46:21.373Z","trace_propagation_headers":{"sentry-trace":"8cf1cd960e4f4812848b9de885176b9c-954da9f4a497497e","baggage":"sentry-trace_id=8cf1cd960e4f4812848b9de885176b9c,sentry-release=Unknown"},"correlation_id":"01KY1T7NVYK15ZY9R6V8BF6YW4","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:21.376Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8e26eab49b3e8b165a0f3187: done: 0.023501 sec","job_status":"done","queue_duration_s":0.078779,"scheduling_latency_s":0.078793,"redis_calls":10,"redis_duration_s":0.0020819999999999996,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000421,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001218,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000270705,"duration_s":0.023501,"completed_at":"2026-07-21T07:46:21.478Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004266,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses gitlab # [ 185.223375] gitaly[1358]: time="2026-07-21T07:46:21.914Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7Q12V0QTT3XA1JJ367FJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:51.812" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:21.913" grpc.time_ms=0.943 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 185.258696] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:21 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 90 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses, in 0.14 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses gitlab # [ 186.352823] gitaly[1358]: time="2026-07-21T07:46:23.043Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7R4E8V6NRS75Q8P60YB1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:53.041" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:23.042" grpc.time_ms=0.983 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 186.434419] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 134 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses, in 0.17 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses gitlab # [ 187.526290] gitaly[1358]: time="2026-07-21T07:46:24.216Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7S9540RMV247X016SN1E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:54.215" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:24.215" grpc.time_ms=0.735 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 187.580867] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 105 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses, in 0.15 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses gitlab # [ 188.675305] gitaly[1358]: time="2026-07-21T07:46:25.365Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7TCZ5HRHPM42GPM7HMZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:55.364" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:25.364" grpc.time_ms=0.865 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses, in 0.11 seconds) gitlab # [ 188.703471] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses HTTP/1.1" 200 633 "" "curl/8.20.0" 79 gitlab # [ 189.604743] nix-daemon[3271]: accepted connection from pid 3308, user nixbot gitlab # [ 189.651852] gitaly[1358]: time="2026-07-21T07:46:26.343Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7VAZ0N6FHE49P88P0ZRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:56.240" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:26.341" grpc.time_ms=1.489 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses gitlab # [ 189.712562] gitaly[1358]: time="2026-07-21T07:46:26.403Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7VAZ0N6FHE49P88P0ZRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T07:46:36.261" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:26.392" grpc.time_ms=11.017 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 189.758301] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:26.448Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"40c183e89257f10bf8ff650e","created_at":"2026-07-21T07:46:26.443Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-90242651ed8c46a3","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.446Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-40c183e89257f10bf8ff650e: start","job_status":"start","queue_duration_s":0.001794,"scheduling_latency_s":0.00184} gitlab # [ 189.785524] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7499edfcc425b7c57426f5a658af2ef46637209f HTTP/1.1" 201 669 "" "python-httpx/0.28.1" 189 gitlab # [ 189.788036] gitaly[1358]: time="2026-07-21T07:46:26.466Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7VFWTKN7XNEBKDZVW32B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:56.465" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:26.465" grpc.time_ms=0.726 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses, in 0.10 seconds) gitlab # [ 189.799947] nixbot[2951]: {"ts": "2026-07-21T07:46:26.461Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7499edfcc425b7c57426f5a658af2ef46637209f \"HTTP/1.1 201 Created\""} gitlab # [ 189.803270] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:26 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses HTTP/1.1" 200 671 "" "curl/8.20.0" 51 gitlab # [ 189.806063] gitaly[1358]: time="2026-07-21T07:46:26.495Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7VH08V6PYECRAZB451DE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:56.394" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:26.495" grpc.time_ms=0.55 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 189.830931] gitaly[1358]: time="2026-07-21T07:46:26.522Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7VH08V6PYECRAZB451DE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T07:46:36.085" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:26.516" grpc.time_ms=5.98 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 189.859928] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7499edfcc425b7c57426f5a658af2ef46637209f HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 86 gitlab # [ 189.860643] nixbot[2951]: {"ts": "2026-07-21T07:46:26.551Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7499edfcc425b7c57426f5a658af2ef46637209f \"HTTP/1.1 201 Created\""} gitlab # [ 190.000711] gitaly[1358]: time="2026-07-21T07:46:26.691Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7VAZ0N6FHE49P88P0ZRN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:26.691" grpc.time_ms=0.629 pid=1358 span.kind=server system=grpc gitlab # [ 190.048928] gitaly[1358]: time="2026-07-21T07:46:26.740Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7VAZ0N6FHE49P88P0ZRN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:26.739" grpc.time_ms=0.848 pid=1358 span.kind=server system=grpc gitlab # [ 190.091224] gitaly[1358]: time="2026-07-21T07:46:26.782Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T7VAZ0N6FHE49P88P0ZRN grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:26.781" grpc.time_ms=0.564 pid=1358 span.kind=server system=grpc gitlab # [ 190.116846] gitaly[1358]: time="2026-07-21T07:46:26.807Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7VAZ0N6FHE49P88P0ZRN diskcache=76a9278a-4042-43d5-9214-110d314cbe17 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-21T07:46:56.549" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:26.650" pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 190.117712] gitaly[1358]: time="2026-07-21T07:46:26.808Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=144 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7281 command.oublock=0 command.real_time_ms=149 command.spawn_token_fork_ms=0 command.system_time_ms=93 command.user_time_ms=51 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7VAZ0N6FHE49P88P0ZRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-21T07:46:56.549" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:26.650" grpc.time_ms=158.042 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 190.195178] gitaly[1358]: time="2026-07-21T07:46:26.886Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7VAZ0N6FHE49P88P0ZRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-21T07:46:56.784" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:26.885" grpc.time_ms=1.153 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 190.223572] gitaly[1358]: time="2026-07-21T07:46:26.914Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7VAZ0N6FHE49P88P0ZRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:56.813" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:26.913" grpc.time_ms=0.743 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 190.286483] gitaly[1358]: time="2026-07-21T07:46:26.977Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7VAZ0N6FHE49P88P0ZRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:56.875" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:26.976" grpc.time_ms=0.853 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 190.313643] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:26.989Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"993cfddc6c129a1692ee2dcc","created_at":"2026-07-21T07:46:26.986Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","duplicate-of":"40c183e89257f10bf8ff650e","job_size_bytes":3,"pid":2039,"job_status":"deduplicated","message":"PipelineProcessWorker JID-993cfddc6c129a1692ee2dcc: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 190.314354] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:26.991Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"40c183e89257f10bf8ff650e","created_at":"2026-07-21T07:46:26.443Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-90242651ed8c46a3","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.446Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2039,"job_status":"rescheduled","message":"PipelineProcessWorker JID-40c183e89257f10bf8ff650e: rescheduled"} gitlab # [ 190.315908] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:26.999Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"40c183e89257f10bf8ff650e","created_at":"2026-07-21T07:46:26.443Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-90242651ed8c46a3","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.446Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-40c183e89257f10bf8ff650e: done: 0.55122 sec","job_status":"done","queue_duration_s":0.001794,"scheduling_latency_s":0.00184,"gitaly_calls":4,"gitaly_duration_s":0.196153,"redis_calls":116,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.026728,"redis_read_bytes":280,"redis_write_bytes":27321,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00126,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.003189,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_feature_flag_calls":26,"redis_feature_flag_duration_s":0.006532,"redis_feature_flag_read_bytes":26,"redis_feature_flag_write_bytes":4583,"redis_queues_calls":25,"redis_queues_duration_s":0.004024,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14465,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.006537,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000308,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000566,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003052,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":851,"db_count":49,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":49,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.11,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.125,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.163391,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.507674858,"duration_s":0.55122,"completed_at":"2026-07-21T07:46:26.999Z","load_balancing_strategy":"primary","db_duration_s":0.170113,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 190.358920] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.001Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"7eb4f8101af17bb0eb386165","created_at":"2026-07-21T07:46:26.638Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.642Z","job_size_bytes":23,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-7eb4f8101af17bb0eb386165: start","job_status":"start","queue_duration_s":0.358901,"scheduling_latency_s":0.358977} gitlab # [ 190.497756] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.188Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"7eb4f8101af17bb0eb386165","created_at":"2026-07-21T07:46:26.638Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.642Z","job_size_bytes":23,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-7eb4f8101af17bb0eb386165: done: 0.186918 sec","job_status":"done","queue_duration_s":0.358901,"scheduling_latency_s":0.358977,"redis_calls":10,"redis_duration_s":0.101663,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000657,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00204,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.098966,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000423517,"duration_s":0.186918,"completed_at":"2026-07-21T07:46:27.188Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015519,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 190.498827] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.189Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"8db5255405a10fc965810380","created_at":"2026-07-21T07:46:26.832Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.837Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-8db5255405a10fc965810380: start","job_status":"start","queue_duration_s":0.352737,"scheduling_latency_s":0.352751} gitlab # [ 190.605192] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.295Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"8db5255405a10fc965810380","created_at":"2026-07-21T07:46:26.832Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.837Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-8db5255405a10fc965810380: done: 0.105803 sec","job_status":"done","queue_duration_s":0.352737,"scheduling_latency_s":0.352751,"redis_calls":10,"redis_duration_s":0.002714,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000419,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001769,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000526,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.079,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.079,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000517664,"duration_s":0.105803,"completed_at":"2026-07-21T07:46:27.295Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.07917,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.607262] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.298Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"3c9b0f1d98543acf6197674c","created_at":"2026-07-21T07:46:26.847Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.852Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-3c9b0f1d98543acf6197674c: start","job_status":"start","queue_duration_s":0.445723,"scheduling_latency_s":0.445752} gitlab # [ 190.653474] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.343Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"3c9b0f1d98543acf6197674c","created_at":"2026-07-21T07:46:26.847Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.852Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-3c9b0f1d98543acf6197674c: done: 0.045864 sec","job_status":"done","queue_duration_s":0.445723,"scheduling_latency_s":0.445752,"redis_calls":10,"redis_duration_s":0.003493,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00066,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001854,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000979,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000469333,"duration_s":0.045864,"completed_at":"2026-07-21T07:46:27.343Z","load_balancing_strategy":"primary","db_duration_s":0.008788,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.658849] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.345Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"19030666e004ef09435dd112","created_at":"2026-07-21T07:46:26.863Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.866Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-19030666e004ef09435dd112: start","job_status":"start","queue_duration_s":0.478919,"scheduling_latency_s":0.478943} gitlab # [ 190.684636] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.375Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"19030666e004ef09435dd112","created_at":"2026-07-21T07:46:26.863Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.866Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-19030666e004ef09435dd112: done: 0.029793 sec","job_status":"done","queue_duration_s":0.478919,"scheduling_latency_s":0.478943,"redis_calls":13,"redis_duration_s":0.003188,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00056,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001441,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000613,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000574,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000269308,"duration_s":0.029793,"completed_at":"2026-07-21T07:46:27.375Z","load_balancing_strategy":"primary","db_duration_s":0.004619,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.726951] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.376Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"a77d028a13555fb997f2e4f8","created_at":"2026-07-21T07:46:26.867Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.870Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-a77d028a13555fb997f2e4f8: start","job_status":"start","queue_duration_s":0.506525,"scheduling_latency_s":0.506538} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses gitlab # [ 190.728381] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.396Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"a77d028a13555fb997f2e4f8","created_at":"2026-07-21T07:46:26.867Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.870Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-a77d028a13555fb997f2e4f8: done: 0.01976 sec","job_status":"done","queue_duration_s":0.506525,"scheduling_latency_s":0.506538,"redis_calls":10,"redis_duration_s":0.002123,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000475,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001216,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000432,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000269029,"duration_s":0.01976,"completed_at":"2026-07-21T07:46:27.396Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004772,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.793326] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.397Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9b0614b234872bb269886981","created_at":"2026-07-21T07:46:26.870Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.873Z","job_size_bytes":19,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9b0614b234872bb269886981: start","job_status":"start","queue_duration_s":0.523739,"scheduling_latency_s":0.523751} gitlab # [ 190.795642] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.486Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9b0614b234872bb269886981","created_at":"2026-07-21T07:46:26.870Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.873Z","job_size_bytes":19,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9b0614b234872bb269886981: done: 0.088421 sec","job_status":"done","queue_duration_s":0.523739,"scheduling_latency_s":0.523751,"redis_calls":11,"redis_duration_s":0.002283,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000413,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001459,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000411,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.058,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000362337,"duration_s":0.088421,"completed_at":"2026-07-21T07:46:27.486Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.062109,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.824892] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.488Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e21f8a3882e920117e85c87b","created_at":"2026-07-21T07:46:26.875Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.878Z","job_size_bytes":66,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e21f8a3882e920117e85c87b: start","job_status":"start","queue_duration_s":0.609382,"scheduling_latency_s":0.609407} gitlab # [ 190.840744] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.514Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"e21f8a3882e920117e85c87b","created_at":"2026-07-21T07:46:26.875Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.878Z","job_size_bytes":66,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-e21f8a3882e920117e85c87b: done: 0.026576 sec","job_status":"done","queue_duration_s":0.609382,"scheduling_latency_s":0.609407,"redis_calls":10,"redis_duration_s":0.001749,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000343,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001024,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000315403,"duration_s":0.026576,"completed_at":"2026-07-21T07:46:27.514Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002758,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.893142] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.515Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"a05ca45e38d4bcb348ad8d86","created_at":"2026-07-21T07:46:26.879Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.880Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a05ca45e38d4bcb348ad8d86: start","job_status":"start","queue_duration_s":0.634994,"scheduling_latency_s":0.635007} gitlab # [ 190.896422] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.532Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"a05ca45e38d4bcb348ad8d86","created_at":"2026-07-21T07:46:26.879Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.880Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a05ca45e38d4bcb348ad8d86: done: 0.017178 sec","job_status":"done","queue_duration_s":0.634994,"scheduling_latency_s":0.635007,"redis_calls":7,"redis_duration_s":0.000903,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017178,"completed_at":"2026-07-21T07:46:27.532Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002024,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.972398] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.533Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"bd8feb27d252149c5f6198e0","created_at":"2026-07-21T07:46:26.924Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.929Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-bd8feb27d252149c5f6198e0: start","job_status":"start","queue_duration_s":0.603585,"scheduling_latency_s":0.603598} gitlab # [ 190.973033] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.555Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"bd8feb27d252149c5f6198e0","created_at":"2026-07-21T07:46:26.924Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.929Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-bd8feb27d252149c5f6198e0: done: 0.022471 sec","job_status":"done","queue_duration_s":0.603585,"scheduling_latency_s":0.603598,"redis_calls":10,"redis_duration_s":0.001471,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000237,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000279,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000275454,"duration_s":0.022471,"completed_at":"2026-07-21T07:46:27.555Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 190.988842] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.557Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3f1ce708a1a94373382cf4e4","created_at":"2026-07-21T07:46:26.930Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.935Z","job_size_bytes":6,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3f1ce708a1a94373382cf4e4: start","job_status":"start","queue_duration_s":0.621622,"scheduling_latency_s":0.621636} gitlab # [ 191.038979] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.573Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3f1ce708a1a94373382cf4e4","created_at":"2026-07-21T07:46:26.930Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.935Z","job_size_bytes":6,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3f1ce708a1a94373382cf4e4: done: 0.016007 sec","job_status":"done","queue_duration_s":0.621622,"scheduling_latency_s":0.621636,"redis_calls":10,"redis_duration_s":0.0021060000000000002,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000458,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001195,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000267073,"duration_s":0.016007,"completed_at":"2026-07-21T07:46:27.573Z","load_balancing_strategy":"primary","db_duration_s":0.002413,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 191.041176] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.574Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"29c85431ef72a21d8794654a","created_at":"2026-07-21T07:46:26.936Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.941Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-29c85431ef72a21d8794654a: start","job_status":"start","queue_duration_s":0.632978,"scheduling_latency_s":0.63299} gitlab # [ 191.055177] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.598Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"29c85431ef72a21d8794654a","created_at":"2026-07-21T07:46:26.936Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.941Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-29c85431ef72a21d8794654a: done: 0.023797 sec","job_status":"done","queue_duration_s":0.632978,"scheduling_latency_s":0.63299,"redis_calls":10,"redis_duration_s":0.002115,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000412,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001284,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000419,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000275454,"duration_s":0.023797,"completed_at":"2026-07-21T07:46:27.598Z","load_balancing_strategy":"primary","db_duration_s":0.007985,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 191.115948] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.599Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"28ba381062b044b60269e624","created_at":"2026-07-21T07:46:26.992Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.993Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-28ba381062b044b60269e624: start","job_status":"start","queue_duration_s":0.605412,"scheduling_latency_s":0.605425} gitlab # [ 191.116967] gitaly[1358]: time="2026-07-21T07:46:27.803Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7WRHYF0E9NTYF5M6FZ4E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:57.700" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:27.800" grpc.time_ms=2.971 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.158617] gitaly[1358]: time="2026-07-21T07:46:27.849Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=200 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=200 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T7VAZ0N6FHE49P88P0ZRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-21T07:46:57.547" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:27.648" grpc.time_ms=201.541 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.180797] gitaly[1358]: time="2026-07-21T07:46:27.872Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7VAZ0N6FHE49P88P0ZRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:57.771" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:27.871" grpc.time_ms=0.57 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.201954] gitaly[1358]: time="2026-07-21T07:46:27.893Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7VAZ0N6FHE49P88P0ZRN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:57.792" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:27.892" grpc.time_ms=0.457 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses, in 0.46 seconds) gitlab # [ 191.211230] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.902Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"28ba381062b044b60269e624","created_at":"2026-07-21T07:46:26.992Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-ce2d7a7795074031","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:26.993Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-28ba381062b044b60269e624: done: 0.302836 sec","job_status":"done","queue_duration_s":0.605412,"scheduling_latency_s":0.605425,"gitaly_calls":3,"gitaly_duration_s":0.208003,"redis_calls":44,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.006902,"redis_read_bytes":98,"redis_write_bytes":6013,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001069,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":20,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.001536,"redis_cache_read_bytes":38,"redis_cache_write_bytes":1784,"redis_queues_calls":2,"redis_queues_duration_s":0.000291,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001282,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000258,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000351,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001046,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":20,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065308,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.285715922,"duration_s":0.302836,"completed_at":"2026-07-21T07:46:27.902Z","load_balancing_strategy":"primary","db_duration_s":0.027076,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 191.218931] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.907Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"7aafa0afc39d432124b375b6","created_at":"2026-07-21T07:46:27.639Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-2d245dac643144dc","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:27.640Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7aafa0afc39d432124b375b6: start","job_status":"start","queue_duration_s":0.267085,"scheduling_latency_s":0.267102} gitlab # [ 191.240395] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:27.931Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"7aafa0afc39d432124b375b6","created_at":"2026-07-21T07:46:27.639Z","trace_propagation_headers":{"sentry-trace":"304bd8e63fea4b8b903c054c9cd30310-2d245dac643144dc","baggage":"sentry-trace_id=304bd8e63fea4b8b903c054c9cd30310,sentry-release=Unknown"},"correlation_id":"01KY1T7VAZ0N6FHE49P88P0ZRN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:27.640Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7aafa0afc39d432124b375b6: done: 0.023833 sec","job_status":"done","queue_duration_s":0.267085,"scheduling_latency_s":0.267102,"redis_calls":7,"redis_duration_s":0.002974,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002393,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000581,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023833,"completed_at":"2026-07-21T07:46:27.931Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005183,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 191.271103] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:27 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 213 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses gitlab # [ 192.354860] gitaly[1358]: time="2026-07-21T07:46:29.045Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7XZXEXRE70AC1T0FRZPD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:46:59.043" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:29.044" grpc.time_ms=0.797 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.415973] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 117 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses, in 0.16 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses gitlab # [ 193.512591] gitaly[1358]: time="2026-07-21T07:46:30.203Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T7Z459SDG78EJWMYF6ZAJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:47:00.101" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:30.202" grpc.time_ms=0.781 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 193.536094] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 77 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses, in 0.12 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses gitlab # [ 194.625730] nix-daemon[3271]: accepted connection from pid 3392, user nixbot gitlab # [ 194.682166] gitaly[1358]: time="2026-07-21T07:46:31.372Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T807144R61FHPX8FFJ01S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:47:01.371" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:31.371" grpc.time_ms=0.833 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 194.687435] gitaly[1358]: time="2026-07-21T07:46:31.378Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T809EF2MMTZ8TSJRNYZ56 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:47:01.277" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:31.377" grpc.time_ms=0.478 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 194.721176] gitaly[1358]: time="2026-07-21T07:46:31.412Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T809EF2MMTZ8TSJRNYZ56 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-07-21T07:46:41.271" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:31.401" grpc.time_ms=10.386 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 194.746901] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses HTTP/1.1" 200 1307 "" "curl/8.20.0" 171 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses, in 0.21 seconds) gitlab # [ 194.762118] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.452Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5dc0c7acc8f89c123214d09b","created_at":"2026-07-21T07:46:31.448Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-3424578e5109474a","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.451Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5dc0c7acc8f89c123214d09b: start","job_status":"start","queue_duration_s":0.001432,"scheduling_latency_s":0.001473} gitlab # [ 194.776552] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:31 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/7499edfcc425b7c57426f5a658af2ef46637209f HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 120 gitlab # [ 194.777130] nixbot[2951]: {"ts": "2026-07-21T07:46:31.463Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/7499edfcc425b7c57426f5a658af2ef46637209f \"HTTP/1.1 201 Created\""} gitlab # [ 194.836773] gitaly[1358]: time="2026-07-21T07:46:31.528Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T809EF2MMTZ8TSJRNYZ56 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:31.527" grpc.time_ms=0.401 pid=1358 span.kind=server system=grpc gitlab # [ 194.864115] gitaly[1358]: time="2026-07-21T07:46:31.555Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T809EF2MMTZ8TSJRNYZ56 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3901 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:31.554" grpc.time_ms=0.363 pid=1358 span.kind=server system=grpc gitlab # [ 194.890593] gitaly[1358]: time="2026-07-21T07:46:31.581Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01KY1T809EF2MMTZ8TSJRNYZ56 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3903 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-07-21T07:46:31.580" grpc.time_ms=0.508 pid=1358 span.kind=server system=grpc gitlab # [ 194.909063] gitaly[1358]: time="2026-07-21T07:46:31.600Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T809EF2MMTZ8TSJRNYZ56 diskcache=56dbbba0-136d-4a24-aec6-5837311f26e9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-21T07:47:01.408" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:31.508" pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 194.909377] gitaly[1358]: time="2026-07-21T07:46:31.600Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=80 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7026 command.oublock=0 command.real_time_ms=82 command.spawn_token_fork_ms=0 command.system_time_ms=46 command.user_time_ms=34 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T809EF2MMTZ8TSJRNYZ56 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-07-21T07:47:01.408" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-07-21T07:46:31.508" grpc.time_ms=91.646 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 194.916208] nix-daemon[3271]: accepted connection from pid 3426, user nixbot gitlab # [ 194.945771] gitaly[1358]: time="2026-07-21T07:46:31.636Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T809EF2MMTZ8TSJRNYZ56 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-07-21T07:47:01.535" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:31.636" grpc.time_ms=0.656 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 194.977271] gitaly[1358]: time="2026-07-21T07:46:31.668Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T809EF2MMTZ8TSJRNYZ56 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:47:01.566" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:31.667" grpc.time_ms=0.597 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 195.008961] gitaly[1358]: time="2026-07-21T07:46:31.700Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01KY1T80KE0N7Z3ZAYMEY9X4KP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:31.694" grpc.time_ms=5.793 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 195.014739] gitaly[1358]: time="2026-07-21T07:46:31.706Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T809EF2MMTZ8TSJRNYZ56 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:47:01.605" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:31.705" grpc.time_ms=0.489 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 195.033523] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 61 gitlab # [ 195.041820] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.732Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5dc0c7acc8f89c123214d09b","created_at":"2026-07-21T07:46:31.448Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-3424578e5109474a","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.451Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5dc0c7acc8f89c123214d09b: done: 0.280162 sec","job_status":"done","queue_duration_s":0.001432,"scheduling_latency_s":0.001473,"gitaly_calls":4,"gitaly_duration_s":0.142078,"redis_calls":83,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.011372,"redis_read_bytes":173,"redis_write_bytes":20927,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000714,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":21,"redis_cache_allowed_cross_slot_calls":2,"redis_cache_duration_s":0.002181,"redis_cache_read_bytes":39,"redis_cache_write_bytes":1840,"redis_queues_calls":23,"redis_queues_duration_s":0.002101,"redis_queues_read_bytes":25,"redis_queues_write_bytes":13380,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.00358,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3094,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000157,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000742,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001183,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":36,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":21,"db_ci_count":15,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.103768,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.25947851,"duration_s":0.280162,"completed_at":"2026-07-21T07:46:31.732Z","load_balancing_strategy":"primary","db_duration_s":0.032374,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 195.064840] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.738Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"36c3babb22bd0f50d5609b18","created_at":"2026-07-21T07:46:31.503Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.505Z","job_size_bytes":23,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-36c3babb22bd0f50d5609b18: start","job_status":"start","queue_duration_s":0.23279,"scheduling_latency_s":0.232807} gitlab # [ 195.070924] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.762Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"36c3babb22bd0f50d5609b18","created_at":"2026-07-21T07:46:31.503Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.505Z","job_size_bytes":23,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-36c3babb22bd0f50d5609b18: done: 0.023895 sec","job_status":"done","queue_duration_s":0.23279,"scheduling_latency_s":0.232807,"redis_calls":7,"redis_duration_s":0.001212,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023895,"completed_at":"2026-07-21T07:46:31.761Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005729,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 195.105328] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.763Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"3fe592f2da5d11e1811a5019","created_at":"2026-07-21T07:46:31.605Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.607Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-3fe592f2da5d11e1811a5019: start","job_status":"start","queue_duration_s":0.155988,"scheduling_latency_s":0.156004} gitlab # [ 195.121051] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.776Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"3fe592f2da5d11e1811a5019","created_at":"2026-07-21T07:46:31.605Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.607Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-3fe592f2da5d11e1811a5019: done: 0.013487 sec","job_status":"done","queue_duration_s":0.155988,"scheduling_latency_s":0.156004,"redis_calls":7,"redis_duration_s":0.000925,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000661,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000264,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013487,"completed_at":"2026-07-21T07:46:31.776Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002527,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.192635] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.777Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c3386730f99bbe71e83878a9","created_at":"2026-07-21T07:46:31.611Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.613Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c3386730f99bbe71e83878a9: start","job_status":"start","queue_duration_s":0.163512,"scheduling_latency_s":0.163526} gitlab # [ 195.193645] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.851Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c3386730f99bbe71e83878a9","created_at":"2026-07-21T07:46:31.611Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.613Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c3386730f99bbe71e83878a9: done: 0.074424 sec","job_status":"done","queue_duration_s":0.163512,"scheduling_latency_s":0.163526,"redis_calls":7,"redis_duration_s":0.001024,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000754,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.059,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.074424,"completed_at":"2026-07-21T07:46:31.851Z","load_balancing_strategy":"primary","db_duration_s":0.059399,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.216738] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.852Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"d2c78d5131d6641e4477a676","created_at":"2026-07-21T07:46:31.618Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.619Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-d2c78d5131d6641e4477a676: start","job_status":"start","queue_duration_s":0.233178,"scheduling_latency_s":0.233198} gitlab # [ 195.231714] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.868Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"d2c78d5131d6641e4477a676","created_at":"2026-07-21T07:46:31.618Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.619Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-d2c78d5131d6641e4477a676: done: 0.015752 sec","job_status":"done","queue_duration_s":0.233178,"scheduling_latency_s":0.233198,"redis_calls":10,"redis_duration_s":0.0012439999999999999,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000646,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000359,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000239,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015752,"completed_at":"2026-07-21T07:46:31.868Z","load_balancing_strategy":"primary","db_duration_s":0.002027,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.278790] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.869Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"f1401022fe671dad74f47e28","created_at":"2026-07-21T07:46:31.620Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.620Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-f1401022fe671dad74f47e28: start","job_status":"start","queue_duration_s":0.248398,"scheduling_latency_s":0.248411} gitlab # [ 195.288867] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.881Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"f1401022fe671dad74f47e28","created_at":"2026-07-21T07:46:31.620Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.620Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-f1401022fe671dad74f47e28: done: 0.012041 sec","job_status":"done","queue_duration_s":0.248398,"scheduling_latency_s":0.248411,"redis_calls":7,"redis_duration_s":0.0008799999999999999,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000641,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000239,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012041,"completed_at":"2026-07-21T07:46:31.881Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002059,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.364241] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.882Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4a7de841d3f92e6c04213093","created_at":"2026-07-21T07:46:31.621Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.623Z","job_size_bytes":19,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4a7de841d3f92e6c04213093: start","job_status":"start","queue_duration_s":0.258254,"scheduling_latency_s":0.258266} gitlab # [ 195.365064] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.900Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"4a7de841d3f92e6c04213093","created_at":"2026-07-21T07:46:31.621Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.623Z","job_size_bytes":19,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-4a7de841d3f92e6c04213093: done: 0.017955 sec","job_status":"done","queue_duration_s":0.258254,"scheduling_latency_s":0.258266,"redis_calls":8,"redis_duration_s":0.001161,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000301,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017955,"completed_at":"2026-07-21T07:46:31.900Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001821,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.384779] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.901Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c4a0c62251eb0677bfb4d689","created_at":"2026-07-21T07:46:31.624Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.626Z","job_size_bytes":66,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c4a0c62251eb0677bfb4d689: start","job_status":"start","queue_duration_s":0.274879,"scheduling_latency_s":0.274895} gitlab # [ 195.433110] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.911Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"c4a0c62251eb0677bfb4d689","created_at":"2026-07-21T07:46:31.624Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.626Z","job_size_bytes":66,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-c4a0c62251eb0677bfb4d689: done: 0.010554 sec","job_status":"done","queue_duration_s":0.274879,"scheduling_latency_s":0.274895,"redis_calls":7,"redis_duration_s":0.000919,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000665,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000254,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010554,"completed_at":"2026-07-21T07:46:31.911Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000795,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.442760] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.912Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"334c914f979316cb71ba9efe","created_at":"2026-07-21T07:46:31.627Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.630Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-334c914f979316cb71ba9efe: start","job_status":"start","queue_duration_s":0.282395,"scheduling_latency_s":0.282408} gitlab # [ 195.458777] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.929Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"334c914f979316cb71ba9efe","created_at":"2026-07-21T07:46:31.627Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.630Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-334c914f979316cb71ba9efe: done: 0.016986 sec","job_status":"done","queue_duration_s":0.282395,"scheduling_latency_s":0.282408,"redis_calls":7,"redis_duration_s":0.0009029999999999999,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000253,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016986,"completed_at":"2026-07-21T07:46:31.929Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001928,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.540675] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.930Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"85116379cd3b18422423a668","created_at":"2026-07-21T07:46:31.682Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.683Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-85116379cd3b18422423a668: start","job_status":"start","queue_duration_s":0.246422,"scheduling_latency_s":0.246436} gitlab # [ 195.541574] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.944Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"85116379cd3b18422423a668","created_at":"2026-07-21T07:46:31.682Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.683Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-85116379cd3b18422423a668: done: 0.013924 sec","job_status":"done","queue_duration_s":0.246422,"scheduling_latency_s":0.246436,"redis_calls":7,"redis_duration_s":0.0010379999999999999,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00025,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013924,"completed_at":"2026-07-21T07:46:31.944Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002323,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.557536] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.945Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"01d5cf923919897720c27997","created_at":"2026-07-21T07:46:31.684Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.685Z","job_size_bytes":6,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-01d5cf923919897720c27997: start","job_status":"start","queue_duration_s":0.26029,"scheduling_latency_s":0.260308} gitlab # [ 195.572752] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.954Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"01d5cf923919897720c27997","created_at":"2026-07-21T07:46:31.684Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.685Z","job_size_bytes":6,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-01d5cf923919897720c27997: done: 0.009134 sec","job_status":"done","queue_duration_s":0.26029,"scheduling_latency_s":0.260308,"redis_calls":7,"redis_duration_s":0.0008719999999999999,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000236,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.009134,"completed_at":"2026-07-21T07:46:31.954Z","load_balancing_strategy":"primary","db_duration_s":0.00028,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.627408] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.955Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"1af08fdae000dd3a4af8f9b7","created_at":"2026-07-21T07:46:31.685Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.686Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-1af08fdae000dd3a4af8f9b7: start","job_status":"start","queue_duration_s":0.269161,"scheduling_latency_s":0.269174} gitlab # [ 195.628325] sidekiq-cluster[2039]: {"severity":"INFO","time":"2026-07-21T07:46:31.966Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"1af08fdae000dd3a4af8f9b7","created_at":"2026-07-21T07:46:31.685Z","trace_propagation_headers":{"sentry-trace":"d8451badf76543b2a6ef241038cb24aa-d4f2199b2c0e4848","baggage":"sentry-trace_id=d8451badf76543b2a6ef241038cb24aa,sentry-release=Unknown"},"correlation_id":"01KY1T809EF2MMTZ8TSJRNYZ56","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-07-21T07:46:31.686Z","job_size_bytes":3,"pid":2039,"sidekiq_tid":"m1m7","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-1af08fdae000dd3a4af8f9b7: done: 0.010792 sec","job_status":"done","queue_duration_s":0.269161,"scheduling_latency_s":0.269174,"redis_calls":7,"redis_duration_s":0.000848,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000621,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000227,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010792,"completed_at":"2026-07-21T07:46:31.965Z","load_balancing_strategy":"primary","db_duration_s":0.001175,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 195.738071] gitaly[1358]: time="2026-07-21T07:46:32.428Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01KY1T80NFRK87YPZ98QHCRND2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:32.417" pid=1358 remote_ip="2001:db8:1::1" request_sha=cc69fbe136984084848b4359b2caac0b76309173 response_bytes=67 span.kind=server system=grpc gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses gitlab # [ 195.738798] gitaly[1358]: time="2026-07-21T07:46:32.428Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T80NFRK87YPZ98QHCRND2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:32.417" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 195.739571] gitlab-workhorse[1289]: time="2026-07-21T07:46:32Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 195.740554] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:32 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.54.0" 701 gitlab # [ 195.741400] gitaly[1358]: time="2026-07-21T07:46:32.429Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=643 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T80NFRK87YPZ98QHCRND2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:32.417" grpc.time_ms=11.782 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.54.0-Linux negotiation.wants=0 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 195.806274] nixbot[2951]: {"ts": "2026-07-21T07:46:32.496Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "cefc112e8680bb0f9569544b9656644ac3312d93"} gitlab # [ 195.876902] gitaly[1358]: time="2026-07-21T07:46:32.568Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T81D4VZ9NRG0KNNV9A15G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:47:02.567" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:32.567" grpc.time_ms=0.516 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 195.898742] gitaly[1358]: time="2026-07-21T07:46:32.589Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=4 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01KY1T81DMP75T97A4QB491713 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-07-21T07:46:32.583" grpc.time_ms=6.149 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 195.899444] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:32 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.54.0" 89 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses, in 0.17 seconds) gitlab # [ 195.918529] gitlab-workhorse[1289]: gitlab 2001:db8:1::1 - - [2026/07/21:07:46:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/7499edfcc425b7c57426f5a658af2ef46637209f/statuses HTTP/1.1" 200 1328 "" "curl/8.20.0" 124 (finished: subtest: eval and build post commit statuses, in 11.93 seconds) (finished: run the VM test script, in 247.17 seconds) gitlab # [ 195.941911] gitaly[1358]: time="2026-07-21T07:46:32.632Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01KY1T81DPTQFSAWEVHYA13VBA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-07-21T07:47:02.532" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-07-21T07:46:32.632" grpc.time_ms=0.411 pid=1358 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root test script finished in 247.20s cleanup kill QemuMachine (pid 13) gitlab # qemu-system-x86_64: terminating on signal 15 from pid 6 (/nix/store/60m4rxhg2fldqaak400c0lry96ijrzqn-python3-3.13.13/bin/python3.13) (finished: cleanup, in 1.41 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [] [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-07-21T07:46:17.609Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [{'id': 1, 'project_id': 1, 'number': 1, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-07-21T07:46:20.663184Z', 'started_at': None, 'finished_at': None}] [] [{'id': 1, 'sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluating flake', 'created_at': '2026-07-21T07:46:21.215Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluating flake', 'created_at': '2026-07-21T07:46:21.215Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluating flake', 'created_at': '2026-07-21T07:46:21.215Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluating flake', 'created_at': '2026-07-21T07:46:21.215Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-21T07:46:21.215Z', 'started_at': None, 'finished_at': '2026-07-21T07:46:26.433Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-21T07:46:21.215Z', 'started_at': None, 'finished_at': '2026-07-21T07:46:26.433Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 2, 'sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'building attributes', 'created_at': '2026-07-21T07:46:26.535Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-21T07:46:21.215Z', 'started_at': None, 'finished_at': '2026-07-21T07:46:26.433Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 2, 'sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'building attributes', 'created_at': '2026-07-21T07:46:26.535Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-21T07:46:21.215Z', 'started_at': None, 'finished_at': '2026-07-21T07:46:26.433Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 2, 'sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'building attributes', 'created_at': '2026-07-21T07:46:26.535Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-21T07:46:21.215Z', 'started_at': None, 'finished_at': '2026-07-21T07:46:26.433Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 2, 'sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'building attributes', 'created_at': '2026-07-21T07:46:26.535Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 1, 'sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': 'evaluation succeeded (1 warning)', 'created_at': '2026-07-21T07:46:21.215Z', 'started_at': None, 'finished_at': '2026-07-21T07:46:26.433Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 2, 'sha': '7499edfcc425b7c57426f5a658af2ef46637209f', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/1', 'description': '1 attributes built', 'created_at': '2026-07-21T07:46:26.535Z', 'started_at': None, 'finished_at': '2026-07-21T07:46:31.440Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 1, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]