tribuchet: building on jamie Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab: QEMU running (pid 46) gitlab # Formatting '/build/vm-state-gitlab/tmp.pPsuWZX6e6', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) gitlab # Discarding device blocks: 0/262144 done gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes (finished: start all VMs, in 50.45 seconds) gitlab # Filesystem UUID: 98c8e95a-ddac-45ca-a340-870b74809977 gitlab: waiting for unit setup-gitlab.service gitlab # Superblock backups stored on blocks: gitlab: waiting for the VM to finish booting gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org) gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BEFD1900+BEF31900 CA00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:03.0)... gitlab # gitlab # gitlab # gitlab # gitlab # iPXE (http://ipxe.org) 00:09.0 CB00 PCI2.10 PnP PMM BEFD1900 BEF31900 CB00 gitlab # Press Ctrl-B to configure iPXE (PCI 00:09.0)... gitlab # gitlab # gitlab # Booting from ROM... gitlab # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rgnvilhdwv9mz4adiif171bhf5cl4ahd-nixos-system-gitlab-test/init regInfo=/nix/store/z54i6c477s9val9pmrlkykkgj84jlchy-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.000000] BIOS-provided physical RAM map: gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffd7fff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000bffd8000-0x00000000bfffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved gitlab # [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001bfffffff] usable gitlab # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved gitlab # [ 0.000000] NX (Execute Disable) protection: active gitlab # [ 0.000000] APIC: Static calls initialized gitlab # [ 0.000000] SMBIOS 2.8 present. gitlab # [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 gitlab # [ 0.000000] DMI: Memory slots populated: 1/1 gitlab # [ 0.000000] Hypervisor detected: KVM gitlab # [ 0.000000] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 gitlab # [ 0.000001] kvm-clock: using sched offset of 477080947 cycles gitlab # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000006] tsc: Detected 2400.012 MHz processor gitlab # [ 0.000813] last_pfn = 0x1c0000 max_arch_pfn = 0x10000000000 gitlab # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs gitlab # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT gitlab # [ 0.000881] last_pfn = 0xbffd8 max_arch_pfn = 0x10000000000 gitlab # [ 0.002740] found SMP MP-table at [mem 0x000f5470-0x000f547f] gitlab # [ 0.002751] Using GB pages for direct mapping gitlab # [ 0.002860] RAMDISK: [mem 0xbe31c000-0xbffcffff] gitlab # [ 0.002869] ACPI: Early table checksum verification disabled gitlab # [ 0.002872] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS ) gitlab # [ 0.002876] ACPI: RSDT 0x00000000BFFE24E3 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002881] ACPI: FACP 0x00000000BFFE237F 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002888] ACPI: DSDT 0x00000000BFFE0040 00233F (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002890] ACPI: FACS 0x00000000BFFE0000 000040 gitlab # [ 0.002891] ACPI: APIC 0x00000000BFFE23F3 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002893] ACPI: HPET 0x00000000BFFE2483 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002894] ACPI: WAET 0x00000000BFFE24BB 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) gitlab # [ 0.002896] ACPI: Reserving FACP table memory at [mem 0xbffe237f-0xbffe23f2] gitlab # [ 0.002897] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe237e] gitlab # [ 0.002897] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] gitlab # [ 0.002898] ACPI: Reserving APIC table memory at [mem 0xbffe23f3-0xbffe2482] gitlab # [ 0.002898] ACPI: Reserving HPET table memory at [mem 0xbffe2483-0xbffe24ba] gitlab # [ 0.002899] ACPI: Reserving WAET table memory at [mem 0xbffe24bb-0xbffe24e2] gitlab # [ 0.003123] No NUMA configuration found gitlab # [ 0.003124] Faking a node at [mem 0x0000000000000000-0x00000001bfffffff] gitlab # [ 0.003127] NODE_DATA(0) allocated [mem 0x1bfff8780-0x1bfffdcff] gitlab # [ 0.005503] Zone ranges: gitlab # [ 0.005504] DMA [mem 0x0000000000001000-0x0000000000ffffff] gitlab # [ 0.005506] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] gitlab # [ 0.005507] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005508] Device empty gitlab # [ 0.005509] Movable zone start for each node gitlab # [ 0.005510] Early memory node ranges gitlab # [ 0.005510] node 0: [mem 0x0000000000001000-0x000000000009efff] gitlab # [ 0.005511] node 0: [mem 0x0000000000100000-0x00000000bffd7fff] gitlab # [ 0.005512] node 0: [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.005513] Initmem setup node 0 [mem 0x0000000000001000-0x00000001bfffffff] gitlab # [ 0.005540] On node 0, zone DMA: 1 pages in unavailable ranges gitlab # [ 0.005815] On node 0, zone DMA: 97 pages in unavailable ranges gitlab # [ 0.118180] On node 0, zone Normal: 40 pages in unavailable ranges gitlab # [ 0.118622] ACPI: PM-Timer IO Port: 0x608 gitlab # [ 0.118637] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) gitlab # [ 0.118674] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 gitlab # [ 0.118677] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) gitlab # [ 0.118679] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) gitlab # [ 0.118680] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) gitlab # [ 0.118681] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) gitlab # [ 0.118682] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) gitlab # [ 0.118684] ACPI: Using ACPI (MADT) for SMP configuration information gitlab # [ 0.118686] ACPI: HPET id: 0x8086a201 base: 0xfed00000 gitlab # [ 0.118690] TSC deadline timer available gitlab # [ 0.118695] CPU topo: Max. logical packages: 1 gitlab # [ 0.118696] CPU topo: Max. logical dies: 1 gitlab # [ 0.118696] CPU topo: Max. dies per package: 1 gitlab # [ 0.118700] CPU topo: Max. threads per core: 1 gitlab # [ 0.118701] CPU topo: Num. cores per package: 4 gitlab # [ 0.118701] CPU topo: Num. threads per package: 4 gitlab # [ 0.118702] CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs gitlab # [ 0.118728] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() gitlab # [ 0.118747] kvm-guest: KVM setup pv remote TLB flush gitlab # [ 0.118749] kvm-guest: setup PV sched yield gitlab # [ 0.118766] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] gitlab # [ 0.118768] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff] gitlab # [ 0.118769] PM: hibernation: Registered nosave memory: [mem 0xbffd8000-0xffffffff] gitlab # [ 0.118770] [mem 0xc0000000-0xfeffbfff] available for PCI devices gitlab # [ 0.118772] Booting paravirtualized kernel on KVM gitlab # [ 0.118776] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns gitlab # [ 0.123239] setup_percpu: NR_CPUS:384 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 gitlab # [ 0.125819] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u524288 gitlab # [ 0.125873] kvm-guest: PV spinlocks enabled gitlab # [ 0.125875] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) gitlab # [ 0.125878] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rgnvilhdwv9mz4adiif171bhf5cl4ahd-nixos-system-gitlab-test/init regInfo=/nix/store/z54i6c477s9val9pmrlkykkgj84jlchy-closure-info/registration console=ttyS0,115200n8 console=tty0 gitlab # [ 0.125978] Unknown kernel command line parameters "regInfo=/nix/store/z54i6c477s9val9pmrlkykkgj84jlchy-closure-info/registration", will be passed to user space. gitlab # [ 0.125991] random: crng init done gitlab # [ 0.125992] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes gitlab # [ 0.135179] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.139634] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.139710] software IO TLB: area num 4. gitlab # [ 0.217632] Fallback order for Node 0: 0 gitlab # [ 0.217641] Built 1 zonelists, mobility grouping on. Total pages: 1572726 gitlab # [ 0.217643] Policy zone: Normal gitlab # [ 0.220576] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.230989] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.244990] allocated 12582912 bytes of page_ext gitlab # [ 0.255258] ftrace: allocating 48717 entries in 192 pages gitlab # [ 0.255260] ftrace: allocated 192 pages with 2 groups gitlab # [ 0.256209] Dynamic Preempt: lazy gitlab # [ 0.256430] rcu: Preemptible hierarchical RCU implementation. gitlab # [ 0.256431] rcu: RCU event tracing is enabled. gitlab # [ 0.256432] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.256434] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.256434] Rude variant of Tasks RCU enabled. gitlab # [ 0.256434] Tracing variant of Tasks RCU enabled. gitlab # [ 0.256435] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. gitlab # [ 0.256436] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.256462] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.256464] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.256465] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.260946] NR_IRQS: 24832, nr_irqs: 456, preallocated irqs: 16 gitlab # [ 0.261282] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.261292] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns gitlab # [ 0.261412] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.265006] Console: colour VGA+ 80x25 gitlab # [ 0.265009] printk: legacy console [tty0] enabled gitlab # [ 0.295823] printk: legacy console [ttyS0] enabled gitlab # [ 0.404853] ACPI: Core revision 20250807 gitlab # [ 0.405728] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns gitlab # [ 0.407331] APIC: Switch to symmetric I/O mode setup gitlab # [ 0.408383] x2apic enabled gitlab # [ 0.409139] APIC: Switched APIC routing to: physical x2apic gitlab # [ 0.410101] kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() gitlab # [ 0.411391] kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() gitlab # [ 0.412893] kvm-guest: setup PV IPIs gitlab # [ 0.414501] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 gitlab # [ 0.415616] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 0.417422] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012) gitlab # [ 0.418507] x86/cpu: User Mode Instruction Prevention (UMIP) activated gitlab # [ 0.419566] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 gitlab # [ 0.421419] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 gitlab # [ 0.422424] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto gitlab # [ 0.423419] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl gitlab # [ 0.424419] Transient Scheduler Attacks: Vulnerable: No microcode gitlab # [ 0.426210] Spectre V2 : Mitigation: Enhanced / Automatic IBRS gitlab # [ 0.427180] Speculative Return Stack Overflow: Mitigation: Safe RET gitlab # [ 0.428238] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization gitlab # [ 0.429427] Spectre V2 : Enabling IBPB for BPF gitlab # [ 0.430420] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier gitlab # [ 0.431419] active return thunk: srso_alias_return_thunk gitlab # [ 0.432440] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' gitlab # [ 0.433419] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' gitlab # [ 0.435419] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' gitlab # [ 0.436419] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask' gitlab # [ 0.437419] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256' gitlab # [ 0.438419] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256' gitlab # [ 0.439419] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' gitlab # [ 0.440420] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 gitlab # [ 0.442217] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64 gitlab # [ 0.443219] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512 gitlab # [ 0.444224] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024 gitlab # [ 0.445218] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8 gitlab # [ 0.446224] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format. gitlab # [ 0.476742] Freeing SMP alternatives memory: 44K gitlab # [ 0.477422] pid_max: default: 32768 minimum: 301 gitlab # [ 0.478527] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.479524] landlock: Up and running. gitlab # [ 0.480171] Yama: becoming mindful. gitlab # [ 0.481098] LSM support for eBPF active gitlab # [ 0.482180] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.483581] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.486577] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1) gitlab # [ 0.488015] Performance Events: Fam17h+ core perfctr, AMD PMU driver. gitlab # [ 0.488428] ... version: 2 gitlab # [ 0.489227] ... bit width: 48 gitlab # [ 0.489421] ... generic counters: 6 gitlab # [ 0.490181] ... generic bitmap: 000000000000003f gitlab # [ 0.490421] ... fixed-purpose counters: 0 gitlab # [ 0.491196] ... fixed-purpose bitmap: 0000000000000000 gitlab # [ 0.491421] ... value mask: 0000ffffffffffff gitlab # [ 0.492382] ... max period: 00007fffffffffff gitlab # [ 0.493183] ... global_ctrl mask: 000000000000003f gitlab # [ 0.493525] signal: max sigframe size: 3376 gitlab # [ 0.494383] rcu: Hierarchical SRCU implementation. gitlab # [ 0.495079] rcu: Max phase no-delay instances is 400. gitlab # [ 0.495620] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.501203] smp: Bringing up secondary CPUs ... gitlab # [ 0.501824] smpboot: x86: Booting SMP configuration: gitlab # [ 0.502425] .... node #0, CPUs: #1 #2 #3 gitlab # [ 0.505569] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.507167] smpboot: Total of 4 processors activated (19200.09 BogoMIPS) gitlab # [ 0.507775] Memory: 6011564K/6290904K available (17210K kernel code, 2726K rwdata, 13576K rodata, 3640K init, 2992K bss, 260108K reserved, 0K cma-reserved) gitlab # [ 0.510421] devtmpfs: initialized gitlab # [ 0.511579] x86/mm: Memory block size: 128MB gitlab # [ 0.514607] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.516512] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.517537] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.518650] PM: RTC time: 19:17:14, date: 2026-09-23 gitlab # [ 0.522510] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.524786] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.525487] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.527497] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.528433] audit: initializing netlink subsys (disabled) gitlab # [ 0.529638] audit: type=2000 audit(1790191034.786:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.529638] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.531422] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.532421] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.533421] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.534421] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.535443] cpuidle: using governor menu gitlab # [ 0.537636] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 gitlab # [ 0.538675] PCI: Using configuration type 1 for base access gitlab # [ 0.539422] PCI: Using configuration type 1 for extended access gitlab # [ 0.540588] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. gitlab # [ 0.542803] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.544335] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.545306] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.546358] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.551121] ACPI: Added _OSI(Module Device) gitlab # [ 0.553428] ACPI: Added _OSI(Processor Device) gitlab # [ 0.555427] ACPI: Added _OSI(Processor Aggregator Device) gitlab # [ 0.562359] ACPI: 1 ACPI AML tables successfully acquired and loaded gitlab # [ 0.571086] ACPI: Interpreter enabled gitlab # [ 0.573463] ACPI: PM: (supports S0 S3 S4 S5) gitlab # [ 0.574428] ACPI: Using IOAPIC for interrupt routing gitlab # [ 0.576593] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug gitlab # [ 0.580427] PCI: Using E820 reservations for host bridge windows gitlab # [ 0.583776] ACPI: Enabled 2 GPEs in block 00 to 0F gitlab # [ 0.596657] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) gitlab # [ 0.598436] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] gitlab # [ 0.602049] acpiphp: Slot [3] registered gitlab # [ 0.603468] acpiphp: Slot [4] registered gitlab # [ 0.604464] acpiphp: Slot [5] registered gitlab # [ 0.606505] acpiphp: Slot [6] registered gitlab # [ 0.607466] acpiphp: Slot [7] registered gitlab # [ 0.609465] acpiphp: Slot [8] registered gitlab # [ 0.610482] acpiphp: Slot [9] registered gitlab # [ 0.611467] acpiphp: Slot [10] registered gitlab # [ 0.613471] acpiphp: Slot [11] registered gitlab # [ 0.614464] acpiphp: Slot [12] registered gitlab # [ 0.616494] acpiphp: Slot [13] registered gitlab # [ 0.617465] acpiphp: Slot [14] registered gitlab # [ 0.618465] acpiphp: Slot [15] registered gitlab # [ 0.619230] acpiphp: Slot [16] registered gitlab # [ 0.620453] acpiphp: Slot [17] registered gitlab # [ 0.621212] acpiphp: Slot [18] registered gitlab # [ 0.621441] acpiphp: Slot [19] registered gitlab # [ 0.622193] acpiphp: Slot [20] registered gitlab # [ 0.623451] acpiphp: Slot [21] registered gitlab # [ 0.624221] acpiphp: Slot [22] registered gitlab # [ 0.624441] acpiphp: Slot [23] registered gitlab # [ 0.625215] acpiphp: Slot [24] registered gitlab # [ 0.626442] acpiphp: Slot [25] registered gitlab # [ 0.627180] acpiphp: Slot [26] registered gitlab # [ 0.627451] acpiphp: Slot [27] registered gitlab # [ 0.628234] acpiphp: Slot [28] registered gitlab # [ 0.628975] acpiphp: Slot [29] registered gitlab # [ 0.629441] acpiphp: Slot [30] registered gitlab # [ 0.630217] acpiphp: Slot [31] registered gitlab # [ 0.631436] PCI host bridge to bus 0000:00 gitlab # [ 0.632164] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] gitlab # [ 0.633365] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] gitlab # [ 0.634350] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] gitlab # [ 0.635421] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] gitlab # [ 0.637421] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 0.638423] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.640714] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.645985] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 conventional PCI endpoint gitlab # [ 0.650558] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 conventional PCI endpoint gitlab # [ 0.657515] pci 0000:00:01.1: BAR 4 [io 0xc220-0xc22f] gitlab # [ 0.659510] pci 0000:00:01.1: BAR 0 [io 0x01f0-0x01f7]: legacy IDE quirk gitlab # [ 0.662430] pci 0000:00:01.1: BAR 1 [io 0x03f6]: legacy IDE quirk gitlab # [ 0.665427] pci 0000:00:01.1: BAR 2 [io 0x0170-0x0177]: legacy IDE quirk gitlab # [ 0.667427] pci 0000:00:01.1: BAR 3 [io 0x0376]: legacy IDE quirk gitlab # [ 0.670885] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300 conventional PCI endpoint gitlab # [ 0.676523] pci 0000:00:01.2: BAR 4 [io 0xc140-0xc15f] gitlab # [ 0.681143] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 conventional PCI endpoint gitlab # [ 0.685128] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI gitlab # [ 0.688442] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB gitlab # [ 0.690869] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint gitlab # [ 0.708649] pci 0000:00:02.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] gitlab # [ 0.710449] pci 0000:00:02.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] gitlab # [ 0.711444] pci 0000:00:02.0: ROM [mem 0xfebc0000-0xfebcffff pref] gitlab # [ 0.712784] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] gitlab # [ 0.716590] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.727466] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f] gitlab # [ 0.730444] pci 0000:00:03.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] gitlab # [ 0.732481] pci 0000:00:03.0: BAR 4 [mem 0xe0000000000-0xe0000003fff 64bit pref] gitlab # [ 0.760448] pci 0000:00:03.0: ROM [mem 0xfeb40000-0xfeb7ffff pref] gitlab # [ 0.765428] pci 0000:00:04.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.771462] pci 0000:00:04.0: BAR 0 [io 0xc180-0xc19f] gitlab # [ 0.773429] pci 0000:00:04.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] gitlab # [ 0.774444] pci 0000:00:04.0: BAR 4 [mem 0xe0000004000-0xe0000007fff 64bit pref] gitlab # [ 0.777070] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.782465] pci 0000:00:05.0: BAR 0 [io 0xc1a0-0xc1bf] gitlab # [ 0.784441] pci 0000:00:05.0: BAR 1 [mem 0xfebd3000-0xfebd3fff] gitlab # [ 0.786480] pci 0000:00:05.0: BAR 4 [mem 0xe0000008000-0xe000000bfff 64bit pref] gitlab # [ 0.792396] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.801557] pci 0000:00:06.0: BAR 0 [io 0xc1c0-0xc1df] gitlab # [ 0.803443] pci 0000:00:06.0: BAR 1 [mem 0xfebd4000-0xfebd4fff] gitlab # [ 0.806483] pci 0000:00:06.0: BAR 4 [mem 0xe000000c000-0xe000000ffff 64bit pref] gitlab # [ 0.811955] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.819465] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f] gitlab # [ 0.820430] pci 0000:00:07.0: BAR 1 [mem 0xfebd5000-0xfebd5fff] gitlab # [ 0.821444] pci 0000:00:07.0: BAR 4 [mem 0xe0000010000-0xe0000013fff 64bit pref] gitlab # [ 0.823718] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.828451] pci 0000:00:08.0: BAR 0 [io 0xc080-0xc0ff] gitlab # [ 0.829394] pci 0000:00:08.0: BAR 1 [mem 0xfebd6000-0xfebd6fff] gitlab # [ 0.831232] pci 0000:00:08.0: BAR 4 [mem 0xe0000014000-0xe0000017fff 64bit pref] gitlab # [ 0.833718] pci 0000:00:09.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.838450] pci 0000:00:09.0: BAR 0 [io 0xc1e0-0xc1ff] gitlab # [ 0.839367] pci 0000:00:09.0: BAR 1 [mem 0xfebd7000-0xfebd7fff] gitlab # [ 0.840248] pci 0000:00:09.0: BAR 4 [mem 0xe0000018000-0xe000001bfff 64bit pref] gitlab # [ 0.841421] pci 0000:00:09.0: ROM [mem 0xfeb80000-0xfebbffff pref] gitlab # [ 0.846839] pci 0000:00:0a.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.856488] pci 0000:00:0a.0: BAR 1 [mem 0xfebd8000-0xfebd8fff] gitlab # [ 0.859489] pci 0000:00:0a.0: BAR 4 [mem 0xe000001c000-0xe000001ffff 64bit pref] gitlab # [ 0.865578] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.874479] pci 0000:00:0b.0: BAR 0 [io 0xc100-0xc13f] gitlab # [ 0.877448] pci 0000:00:0b.0: BAR 1 [mem 0xfebd9000-0xfebd9fff] gitlab # [ 0.879484] pci 0000:00:0b.0: BAR 4 [mem 0xe0000020000-0xe0000023fff 64bit pref] gitlab # [ 0.885903] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.895581] pci 0000:00:0c.0: BAR 0 [io 0xc200-0xc21f] gitlab # [ 0.897440] pci 0000:00:0c.0: BAR 1 [mem 0xfebda000-0xfebdafff] gitlab # [ 0.899469] pci 0000:00:0c.0: BAR 4 [mem 0xe0000024000-0xe0000027fff 64bit pref] gitlab # [ 0.908555] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 gitlab # [ 0.909595] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 gitlab # [ 0.910561] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 gitlab # [ 0.911590] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 gitlab # [ 0.912500] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 gitlab # [ 0.916527] iommu: Default domain type: Translated gitlab # [ 0.917423] iommu: DMA domain TLB invalidation policy: lazy mode gitlab # [ 0.918733] ACPI: bus type USB registered gitlab # [ 0.919568] usbcore: registered new interface driver usbfs gitlab # [ 0.920442] usbcore: registered new interface driver hub gitlab # [ 0.921363] usbcore: registered new device driver usb gitlab # [ 0.923708] NetLabel: Initializing gitlab # [ 0.924345] NetLabel: domain hash size = 128 gitlab # [ 0.925000] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO gitlab # [ 0.926511] NetLabel: unlabeled traffic allowed by default gitlab # [ 0.927432] PCI: Using ACPI for IRQ routing gitlab # [ 0.928717] pci 0000:00:02.0: vgaarb: setting as boot VGA device gitlab # [ 0.929418] pci 0000:00:02.0: vgaarb: bridge control possible gitlab # [ 0.929418] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none gitlab # [ 0.932427] vgaarb: loaded gitlab # [ 0.933209] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 gitlab # [ 0.934187] hpet0: 3 comparators, 64-bit 100.000000 MHz counter gitlab # [ 0.939514] clocksource: Switched to clocksource kvm-clock gitlab # [ 0.947483] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.948319] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.950004] pnp: PnP ACPI init gitlab # [ 0.951258] pnp: PnP ACPI: found 6 devices gitlab # [ 0.961330] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns gitlab # [ 0.963064] clocksource: Switched to clocksource acpi_pm gitlab # [ 0.964081] NET: Registered PF_INET protocol family gitlab # [ 0.966229] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.984475] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.986084] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.987721] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.991562] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.992965] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.994233] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.995885] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.997395] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.998841] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.999933] NET: Registered PF_XDP protocol family gitlab # [ 1.000856] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] gitlab # [ 1.001970] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] gitlab # [ 1.003085] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] gitlab # [ 1.004286] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] gitlab # [ 1.005500] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] gitlab # [ 1.006943] pci 0000:00:01.0: PIIX3: Enabling Passive Release gitlab # [ 1.007997] pci 0000:00:00.0: Limiting direct PCI/PCI transfers gitlab # [ 1.010634] ACPI: \_SB_.LNKD: Enabled at IRQ 11 gitlab # [ 1.013398] PCI: CLS 0 bytes, default 64 gitlab # [ 1.014231] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) gitlab # [ 1.014647] Trying to unpack rootfs image as initramfs... gitlab # [ 1.015384] software IO TLB: mapped [mem 0x00000000ba31c000-0x00000000be31c000] (64MB) gitlab # [ 1.015511] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns gitlab # [ 1.040363] Initialise system trusted keyrings gitlab # [ 1.041569] workingset: timestamp_bits=40 max_order=21 bucket_order=0 gitlab # [ 1.053336] Key type asymmetric registered gitlab # [ 1.054164] Asymmetric key parser 'x509' registered gitlab # [ 1.055108] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) gitlab # [ 1.056854] io scheduler mq-deadline registered gitlab # [ 1.057710] io scheduler kyber registered gitlab # [ 1.060740] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 1.062175] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A gitlab # [ 1.064726] Linux agpgart interface v0.103 gitlab # [ 1.065545] ACPI: bus type drm_connector registered gitlab # [ 1.067945] usbcore: registered new interface driver usbserial_generic gitlab # [ 1.069086] usbserial: USB Serial support registered for generic gitlab # [ 1.070187] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS. gitlab # [ 1.070187] Please enable it if your BIOS has the CPPC option. gitlab # [ 1.072558] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled gitlab # [ 1.074032] drop_monitor: Initializing network drop monitor service gitlab # [ 1.075313] NET: Registered PF_INET6 protocol family gitlab # [ 1.077166] Segment Routing with IPv6 gitlab # [ 1.077872] In-situ OAM (IOAM) with IPv6 gitlab # [ 1.080756] IPI shorthand broadcast: enabled gitlab # [ 1.085106] sched_clock: Marking stable (936011661, 148125989)->(1109983960, -25846310) gitlab # [ 1.086973] registered taskstats version 1 gitlab # [ 1.088054] Loading compiled-in X.509 certificates gitlab # [ 1.097986] Demotion targets for Node 0: null gitlab # [ 1.099161] Key type .fscrypt registered gitlab # [ 1.099919] Key type fscrypt-provisioning registered gitlab # [ 1.100949] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 1.101946] ima: Allocated hash algorithm: sha1 gitlab # [ 1.102810] ima: No architecture policies found gitlab # [ 1.103895] PM: Magic number: 10:483:294 gitlab # [ 1.104654] misc hpet: hash matches gitlab # [ 1.105319] tty tty27: hash matches gitlab # [ 1.106903] RAS: Correctable Errors collector initialized. gitlab # [ 1.111442] clk: Disabling unused clocks gitlab # [ 1.112208] PM: genpd: Disabling unused power domains gitlab # [ 1.310703] Freeing initrd memory: 29392K gitlab # [ 1.318238] Freeing unused decrypted memory: 2028K gitlab # [ 1.324646] Freeing unused kernel image (initmem) memory: 3640K gitlab # [ 1.327391] Write protecting the kernel read-only data: 32768k gitlab # [ 1.332306] Freeing unused kernel image (text/rodata gap) memory: 1220K gitlab # [ 1.336276] Freeing unused kernel image (rodata/data gap) memory: 760K gitlab # [ 1.404258] x86/mm: Checked W+X mappings: passed, no W+X pages found. gitlab # [ 1.405432] Run /init as init process gitlab # [ 1.418494] systemd[1]: Inserted module 'autofs4' gitlab # [ 1.441463] fuse: init (API version 7.45) gitlab # [ 1.449681] ACPI: \_SB_.LNKC: Enabled at IRQ 10 gitlab # [ 1.456796] ACPI: \_SB_.LNKA: Enabled at IRQ 10 gitlab # [ 1.460149] ACPI: \_SB_.LNKB: Enabled at IRQ 11 gitlab # [ 1.503608] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 1.840854] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.881846] systemd[1]: Detected virtualization kvm. gitlab # [ 1.884086] systemd[1]: Detected architecture x86-64. gitlab # [ 1.886309] systemd[1]: Running in initrd. gitlab # [ 1.888881] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.891918] systemd[1]: Hostname set to . gitlab # [ 2.002830] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 2.049310] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 2.061949] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 2.063261] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 2.064742] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 2.065980] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 2.067170] systemd[1]: Reached target Path Units. gitlab # [ 2.068097] systemd[1]: Reached target Slice Units. gitlab # [ 2.069163] systemd[1]: Reached target Swaps. gitlab # [ 2.070012] systemd[1]: Reached target Timer Units. gitlab # [ 2.071156] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 2.072570] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 2.073767] systemd[1]: Listening on Journal Sockets. gitlab # [ 2.074823] systemd[1]: Listening on udev Control Socket. gitlab # [ 2.075935] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 2.076956] systemd[1]: Reached target Socket Units. gitlab # [ 2.079156] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 2.081551] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 2.083803] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 2.100296] systemd[1]: Starting Journal Service... gitlab # [ 2.104238] systemd[1]: Starting Load Kernel Modules... gitlab # [ 2.106276] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 2.111270] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 2.116487] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 2.119937] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 2.126288] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 2.130838] systemd-journald[91]: Collecting audit messages is disabled. gitlab # [ 2.134554] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config gitlab # [ 2.154825] netfs: FS-Cache loaded gitlab # [ 2.156281] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 2.164216] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 2.169412] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 2.171367] 9pnet: Installing 9P2000 support gitlab # [ 2.173859] systemd[1]: Finished Load Kernel Modules. gitlab # [ 2.176463] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 2.183680] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 2.185691] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 2.187589] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 2.189546] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 2.192007] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 2.206459] systemd[1]: Started Journal Service. gitlab # [ 2.058375] systemd-modules-load[94]: Using 4 probe threads gitlab # [ 2.060310] systemd-modules-load[94]: Inserted module 'virtio_balloon' gitlab # [ 2.061442] systemd-modules-load[94]: Inserted module 'virtio_gpu' gitlab # [ 2.062469] systemd-modules-load[94]: Inserted module 'dm_mod' gitlab # [ 2.066258] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 2.068254] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 2.069339] systemd[1]: Reached target Local File Systems. gitlab # [ 2.070170] systemd[1]: Starting Create System Files and Directories... gitlab # [ 2.071092] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 2.092294] systemd[1]: Finished Create System Files and Directories. gitlab # [ 2.098630] systemd-udevd[113]: Using default interface naming scheme 'v261'. gitlab # [ 2.109045] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 2.124877] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.125992] systemd[1]: Reached target System Initialization. gitlab # [ 2.127102] systemd[1]: Reached target Basic System. gitlab # [ 2.371218] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 2.389580] uhci_hcd 0000:00:01.2: UHCI Host Controller gitlab # [ 2.391963] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1 gitlab # [ 2.393211] uhci_hcd 0000:00:01.2: detected 2 ports gitlab # [ 2.394476] uhci_hcd 0000:00:01.2: irq 11, io port 0x0000c140 gitlab # [ 2.395787] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.18 gitlab # [ 2.397027] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 gitlab # [ 2.398108] usb usb1: Product: UHCI Host Controller gitlab # [ 2.399337] usb usb1: Manufacturer: Linux 6.18.43 uhci_hcd gitlab # [ 2.400777] usb usb1: SerialNumber: 0000:00:01.2 gitlab # [ 2.403072] SCSI subsystem initialized gitlab # [ 2.403615] hub 1-0:1.0: USB hub found gitlab # [ 2.404679] hub 1-0:1.0: 2 ports detected gitlab # [ 2.410278] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 2.414013] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 2.417739] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 gitlab # [ 2.421920] serio: i8042 KBD port at 0x60,0x64 irq 1 gitlab # [ 2.422728] serio: i8042 AUX port at 0x60,0x64 irq 12 gitlab # [ 2.425930] virtio_blk virtio5: [vdb] 11844520 512-byte logical blocks (6.06 GB/5.65 GiB) gitlab # [ 2.321121] (udev-worker)[122]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.324117] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.476928] scsi host0: ata_piix gitlab # [ 2.478441] scsi host1: ata_piix gitlab # [ 2.480116] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc220 irq 14 lpm-pol 0 gitlab # [ 2.481536] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc228 irq 15 lpm-pol 0 gitlab # [ 2.334289] (udev-worker)[141]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.336805] (udev-worker)[141]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.486023] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 gitlab # [ 2.353101] systemd-vconsole-setup[145]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.357248] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.360439] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.362712] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.363672] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.364515] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.391900] systemd-fsck[159]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.395598] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.634372] usb 1-1: new full-speed USB device number 2 using uhci_hcd gitlab # [ 2.641589] ata2: found unknown device (class 0) gitlab # [ 2.644506] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100 gitlab # [ 2.648662] scsi 1:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5 gitlab # [ 2.701364] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray gitlab # [ 2.728793] cdrom: Uniform CD-ROM driver Revision: 3.20 gitlab # [ 2.801291] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00 gitlab # [ 2.804352] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10 gitlab # [ 2.807075] usb 1-1: Product: QEMU USB Tablet gitlab # [ 2.808771] usb 1-1: Manufacturer: QEMU gitlab # [ 2.810295] usb 1-1: SerialNumber: 28754-0000:00:01.2-1 gitlab # [ 2.843272] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 2.856841] usbcore: registered new interface driver usbhid gitlab # [ 2.858967] usbhid: USB HID core driver gitlab # [ 2.869308] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input2 gitlab # [ 2.873507] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 gitlab # [ 2.971399] systemd[1]: Mounting /sysroot... gitlab # [ 3.248537] EXT4-fs (vda): mounted filesystem 98c8e95a-ddac-45ca-a340-870b74809977 r/w with ordered data mode. Quota mode: none. gitlab # [ 3.106879] systemd[1]: Mounted /sysroot. gitlab # [ 3.108638] systemd[1]: Reached target Initrd Root File System. gitlab # [ 3.110676] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 3.112600] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 3.114463] systemd[1]: Mounting /sysroot/run... gitlab # [ 3.116453] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.122182] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.126405] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.138151] systemd[1]: Mounted /sysroot/run. gitlab # [ 3.287426] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.139814] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.292290] 9p: Installing v9fs 9p2000 file system support gitlab # [ 3.145909] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.147669] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.153086] systemd-sysroot-fstab-check[178]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.158289] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 3.159892] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.161156] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.162960] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.164599] systemd[1]: Reload requested from client PID 178 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.167348] systemd[1]: Reloading... gitlab # [ 3.220739] systemd[1]: Reloading finished in 55 ms. gitlab # [ 3.246342] systemd-sysroot-fstab-check[178]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.249250] systemd-sysroot-fstab-check[178]: Requesting swap.target/start/replace... gitlab # [ 3.251985] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.254271] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.256622] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.258913] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.290391] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.292925] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.972179] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 4.020850] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 4.022912] systemd[1]: Reached target Initrd File Systems. gitlab # [ 4.025120] systemd[1]: Starting Find NixOS closure... gitlab # [ 4.027158] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 4.061975] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 4.065074] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 4.079994] systemd[1]: Finished Find NixOS closure. gitlab # [ 4.082053] systemd[1]: Reached target Initrd Default Target. gitlab # [ 4.084252] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 4.113253] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 4.115464] systemd[1]: Stopped target Basic System. gitlab # [ 4.117423] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 4.119465] systemd[1]: Stopped target Path Units. gitlab # [ 4.121366] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.125282] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.128069] systemd[1]: Stopped target Slice Units. gitlab # [ 4.129943] systemd[1]: Stopped target Socket Units. gitlab # [ 4.131879] systemd[1]: Stopped target System Initialization. gitlab # [ 4.134208] systemd[1]: Stopped target Swaps. gitlab # [ 4.137226] systemd[1]: Stopped target Timer Units. gitlab # [ 4.141224] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.144728] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.147300] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.150446] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.152508] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 4.155765] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.157490] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.159379] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.160967] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.163232] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.164712] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.166785] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.168812] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.170744] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.172384] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.173831] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.175563] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.176713] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.177590] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.178741] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.180115] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.180952] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.181990] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.183082] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.184201] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.185151] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.186197] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.187259] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.188329] systemd[1]: Closed udev Control Socket. gitlab # [ 4.189151] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.190054] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.191194] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.192157] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.193270] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.194227] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.195155] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.200914] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.201956] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.202707] systemd[1]: Reached target Switch Root. gitlab # [ 4.203961] systemd[1]: Starting NixOS Activation... gitlab # [ 4.307755] initrd-nixos-activation-start[226]: booting system configuration /nix/store/rgnvilhdwv9mz4adiif171bhf5cl4ahd-nixos-system-gitlab-test gitlab # [ 4.347421] initrd-nixos-activation-start[226]: running activation script... gitlab # [ 4.572593] initrd-nixos-activation-start[249]: setting up /etc... gitlab # [ 4.683337] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.684634] systemd[1]: Finished NixOS Activation. gitlab # [ 4.687570] systemd[1]: Starting Switch Root... gitlab # [ 4.721178] systemd[1]: Switching root. gitlab # [ 4.933316] systemd-journald[91]: Received SIGTERM from PID 1 (systemd). gitlab # [ 5.039186] NET: Registered PF_VSOCK protocol family gitlab # [ 5.408173] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.417278] systemd[1]: Detected virtualization kvm. gitlab # [ 5.419088] systemd[1]: Detected architecture x86-64. gitlab # [ 5.421076] systemd[1]: Detected first boot. gitlab # [ 5.423640] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.572745] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.648430] systemd[1]: Applying preset policy. gitlab # [ 5.777669] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 5.897697] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 5.899154] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 5.901602] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 5.903838] systemd[1]: Created slice Slice /system/getty. gitlab # [ 5.905286] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 5.906742] systemd[1]: Created slice User and Session Slice. gitlab # [ 5.907788] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 5.909014] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 5.910185] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 5.910955] systemd[1]: Expecting device /dev/ttyS0... gitlab # [ 5.911773] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 5.912664] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 5.913462] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 5.914328] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 5.915226] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 5.916168] systemd[1]: Reached target Path Units. gitlab # [ 5.916907] systemd[1]: Reached target Remote File Systems. gitlab # [ 5.917818] systemd[1]: Reached target Slice Units. gitlab # [ 5.918596] systemd[1]: Reached target Swaps. gitlab # [ 5.920000] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 5.922001] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 5.923522] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 5.925255] systemd[1]: Listening on Factory Reset Management. gitlab # [ 5.926228] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 5.928627] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 5.930080] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 5.932037] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 5.933297] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 5.934516] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.935903] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 5.938797] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 5.939982] systemd[1]: Listening on udev Control Socket. gitlab # [ 5.940996] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 5.943889] systemd[1]: Mounting Huge Pages File System... gitlab # [ 5.946017] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 5.948159] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 5.950256] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 5.953402] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 5.956051] systemd[1]: Load Kernel Module 9pnet_virtio skipped, unmet condition check ConditionKernelModuleLoaded=!9pnet_virtio gitlab # [ 5.963008] systemd[1]: Starting Load Kernel Module configfs... gitlab # [ 5.964176] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 5.965647] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 5.980208] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 5.984260] systemd[1]: Mounting FUSE Control File System... gitlab # [ 5.985225] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 5.989598] systemd[1]: Starting Journal Service... gitlab # [ 5.993416] systemd[1]: Starting Load Kernel Modules... gitlab # [ 5.998200] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 6.001297] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 6.003379] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.011066] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 6.020253] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 6.021566] systemd[1]: Mounted Huge Pages File System. gitlab # [ 6.022937] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 6.024371] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 6.025474] systemd-journald[320]: Collecting audit messages is enabled. gitlab # [ 6.025637] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 6.028602] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 6.030346] systemd[1]: modprobe@configfs.service: Deactivated successfully. gitlab # [ 6.031148] EXT4-fs (vda): re-mounted 98c8e95a-ddac-45ca-a340-870b74809977. gitlab # [ 6.032211] systemd[1]: Finished Load Kernel Module configfs. gitlab # [ 6.034184] systemd[1]: Mounted FUSE Control File System. gitlab # [ 5.892531] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 5.893986] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 5.895332] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 6.044747] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 5.898605] systemd-modules-load[321]: Using 3 probe threads gitlab # [ 5.899839] systemd-modules-load[321]: Inserted module 'tls' gitlab # [ 6.051061] loop: module loaded gitlab # [ 6.053222] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 5.905581] systemd-modules-load[321]: Inserted module 'loop' gitlab # [ 5.910515] systemd-oomd[322]: No swap; memory pressure usage will be degraded gitlab # [ 6.060158] systemd[1]: Started Journal Service. gitlab # [ 5.913809] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 5.914993] systemd[1]: Finished Load Kernel Modules. gitlab # [ 5.915982] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 5.917756] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 5.921778] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 5.923275] systemd[1]: Starting Firewall... gitlab # [ 5.926078] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 5.927381] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 5.938172] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 6.089946] systemd-journald[320]: Received client request to flush runtime journal. gitlab # [ 5.982617] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 5.984648] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.986165] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 5.987572] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 5.988790] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 5.989600] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 5.990333] systemd[1]: Reached target First Boot Complete. gitlab # [ 5.991147] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 5.992127] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 5.993129] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 5.994207] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 6.007315] systemd-udevd[353]: Using default interface naming scheme 'v261'. gitlab # [ 6.011784] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 6.040366] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.107714] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse gitlab # [ 6.126673] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 6.150626] systemd[1]: Condition check resulted in /dev/ttyS0 being skipped. gitlab # [ 6.157373] (udev-worker)[414]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.188745] systemd[1]: Finished Firewall. gitlab # [ 6.192962] (udev-worker)[432]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 6.195489] (udev-worker)[432]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.352547] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 gitlab # [ 6.352622] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 gitlab # [ 6.357237] i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD gitlab # [ 6.360196] ACPI: button: Power Button [PWRF] gitlab # [ 6.367585] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console gitlab # [ 6.367646] rtc_cmos 00:05: RTC can wake from S4 gitlab # [ 6.370399] Floppy drive(s): fd0 is 2.88M AMI BIOS gitlab # [ 6.370517] parport_pc 00:03: reported by Plug and Play ACPI gitlab # [ 6.370623] parport0: PC-style at 0x378, irq 7 [PCSPP(,...)] gitlab # [ 6.375078] rtc_cmos 00:05: registered as rtc0 gitlab # [ 6.375182] rtc_cmos 00:05: setting system clock to 2026-09-23T19:17:19 UTC (1790191039) gitlab # [ 6.375605] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs gitlab # [ 6.378435] systemd-journald[320]: Time jumped backwards, rotating. gitlab # [ 6.382616] Console: switching to colour dummy device 80x25 gitlab # [ 6.384914] FDC 0 is a S82078B gitlab # [ 6.391264] [drm] Found bochs VGA, ID 0xb0c5. gitlab # [ 6.391735] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebd0000. gitlab # [ 6.246336] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 6.400713] bochs-drm 0000:00:02.0: [drm] Registered 1 planes with drm panic gitlab # [ 6.401603] [drm] Initialized bochs-drm 1.0.0 for 0000:00:02.0 on minor 0 gitlab # [ 6.254888] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.256968] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.264203] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.272182] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.274087] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.409963] mousedev: PS/2 mouse device common for all mice gitlab # [ 6.426259] Console: switching to colour frame buffer device 160x50 gitlab # [ 6.429314] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device gitlab # [ 6.431347] input: QEMU Virtio Keyboard as /devices/pci0000:00/0000:00:0a.0/virtio7/input/input4 gitlab # [ 6.448483] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input6 gitlab # [ 6.455001] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input5 gitlab # [ 6.337904] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.348937] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.350178] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.351132] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.513045] erofs (device vdb): failed to read inode meta block (nid: 128505959): -4 gitlab # [ 6.366568] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 6.367659] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 6.386060] systemd[1]: Starting Virtual Console Setup... gitlab # [ 6.552732] ppdev: user-space parallel port driver gitlab # [ 6.573506] kvm_amd: TSC scaling supported gitlab # [ 6.574034] kvm_amd: Nested Virtualization enabled gitlab # [ 6.574722] kvm_amd: Nested Paging enabled gitlab # [ 6.575306] kvm_amd: LBR virtualization supported gitlab # [ 6.575897] kvm_amd: Virtual VMLOAD VMSAVE supported gitlab # [ 6.576609] kvm_amd: Virtual GIF supported gitlab # [ 6.577197] kvm_amd: Virtual NMI enabled gitlab # [ 6.609328] EDAC MC: Ver: 3.0.0 gitlab # [ 6.471893] systemd-vconsole-setup[495]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 6.474326] systemd[1]: Finished Virtual Console Setup. gitlab # [ 6.754819] systemd[1]: Mounting /run/wrappers... gitlab # [ 6.776139] systemd[1]: Mounted /run/wrappers. gitlab # [ 6.778211] systemd[1]: Reached target Local File Systems. gitlab # [ 6.780199] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 6.782534] systemd[1]: Starting register-nix-paths.service... gitlab # [ 6.784982] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 6.787165] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.790192] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 6.795355] systemd[1]: Starting Create System Files and Directories... gitlab # [ 6.824165] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 6.828796] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 6.843148] systemd[1]: Finished Create System Files and Directories. gitlab # [ 6.846753] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 6.849952] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 6.872332] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 6.882033] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 6.883884] systemd[1]: Starting Update is Completed... gitlab # [ 6.898880] systemd[1]: Finished Update is Completed. gitlab # [ 6.962896] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 6.964792] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.188222] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.190319] systemd[1]: Reached target System Initialization. gitlab # [ 7.192145] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.194315] systemd[1]: Started logrotate.timer. gitlab # [ 7.195780] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.197779] systemd[1]: Reached target Timer Units. gitlab # [ 7.199301] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.201242] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.203393] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.205161] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.207751] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.209563] systemd[1]: Reached target Socket Units. gitlab # [ 7.211111] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.277826] dbus-broker-launch[577]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.281073] dbus-broker-launch[577]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.283922] dbus-broker-launch[577]: Invalid user-name in /nix/store/37yal9w16hfdxjh64sww8gn81f4hma6i-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.289434] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.305491] dbus-broker-launch[577]: Ready gitlab # [ 7.309798] systemd[1]: Reached target Basic System. gitlab # [ 7.312870] systemd[1]: Started backdoor.service. gitlab # [ 7.316206] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.318604] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.336151] systemd[1]: Starting Logrotate configuration check... gitlab # [ 7.339822] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.348225] systemd[1]: Starting Post-Boot Actions... gitlab # connecting to host... gitlab # [ 7.360404] systemd[1]: Started Reset console on configuration changes. gitlab # [ 7.367472] logrotate[582]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 7.367670] logrotate[582]: reading config file /nix/store/y3500wq742i80l631mh8nc8rchpsfb9q-logrotate.conf gitlab: connected to guest root shell gitlab # [ 7.368587] logrotate[582]: note: 'monthly' overrides previously specified 'weekly' gitlab: (connecting took 8.00 seconds) gitlab # [ 7.369613] logrotate[582]: Reading state from file: /var/lib/logrotate.status gitlab: (finished: waiting for the VM to finish booting, in 8.00 seconds) gitlab # [ 7.371502] logrotate[582]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.372707] logrotate[582]: Allocating hash table for state file, size 64 entries gitlab # [ 7.374381] logrotate[582]: Handling 4 logs gitlab # [ 7.375590] logrotate[582]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.379580] logrotate[582]: considering log /var/log/btmp gitlab # [ 7.380423] logrotate[582]: Creating new state gitlab # [ 7.380978] logrotate[582]: Now: 2026-09-23 19:17 gitlab # [ 7.381924] logrotate[582]: Last rotated at 2026-09-23 19:00 gitlab # [ 7.383068] logrotate[582]: log does not need rotating (log has already been rotated) gitlab # [ 7.383934] logrotate[582]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.385821] logrotate[582]: considering log /var/log/wtmp gitlab # [ 7.386945] logrotate[582]: Creating new state gitlab # [ 7.387529] logrotate[582]: Now: 2026-09-23 19:17 gitlab # [ 7.388028] logrotate[582]: Last rotated at 2026-09-23 19:00 gitlab # [ 7.388532] logrotate[582]: log does not need rotating (log has already been rotated) gitlab # [ 7.389076] logrotate[582]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 7.389418] logrotate[582]: switching euid from 0 to 165 and egid from 0 to 165 (pid 582) gitlab # [ 7.389928] logrotate[582]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.390647] logrotate[582]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.391128] logrotate[582]: Creating new state gitlab # [ 7.391614] logrotate[582]: switching euid from 165 to 0 and egid from 165 to 0 (pid 582) gitlab # [ 7.392228] logrotate[582]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.392719] logrotate[582]: switching euid from 0 to 60 and egid from 0 to 60 (pid 582) gitlab # [ 7.393387] logrotate[582]: considering log /var/log/nginx/*.log gitlab # [ 7.393836] logrotate[582]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.394503] logrotate[582]: Creating new state gitlab # [ 7.395153] logrotate[582]: not running postrotate script, since no logs were rotated gitlab # [ 7.396396] logrotate[582]: switching euid from 60 to 0 and egid from 60 to 0 (pid 582) gitlab # [ 7.397257] systemd[1]: Starting resolvconf update... gitlab # [ 7.397939] nsncd[583]: Sep 23 19:17:20.659 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.398866] systemd[1]: Finished Logrotate configuration check. gitlab # [ 7.399906] systemd[1]: Finished Post-Boot Actions. gitlab # [ 7.400837] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 7.401565] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.421885] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.422211] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.426087] systemd[1]: Starting User Login Management... gitlab # [ 7.467599] systemd-logind[621]: New seat seat0. gitlab # [ 7.470720] systemd-logind[621]: Watching system buttons on /dev/input/event2 (Power Button) gitlab # [ 7.471115] systemd-logind[621]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 7.472221] systemd-logind[621]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) gitlab # [ 7.472893] systemd[1]: Started User Login Management. gitlab # [ 7.474418] systemd[1]: Starting linger-users.service... gitlab # [ 7.478893] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.479380] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.480034] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.480520] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.482760] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 7.483951] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 7.484799] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 7.486839] systemd[1]: nscd.service: Consumed 23ms CPU time over 144ms wall clock time, 7.4M memory peak, 5.5M read from disk. gitlab # [ 7.490840] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.495370] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 7.497139] systemd[1]: Finished linger-users.service. gitlab # [ 7.507497] nsncd[676]: Sep 23 19:17:20.779 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.508067] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.508660] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.509170] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.517653] systemd[1]: Finished resolvconf update. gitlab # [ 7.518650] systemd[1]: Reached target Preparation for Network. gitlab # [ 7.522257] systemd[1]: Starting DHCP Client... gitlab # [ 7.525069] systemd[1]: Starting Address configuration of eth1... gitlab # [ 7.526347] systemd[1]: Starting Extra networking commands.... gitlab # [ 7.551776] network-addresses-eth1-start[700]: adding address 192.168.1.1/24... done gitlab # [ 7.555980] network-addresses-eth1-start[700]: adding address 2001:db8:1::1/64... done gitlab # [ 7.562728] systemd[1]: Finished Address configuration of eth1. gitlab # [ 7.585926] systemd[1]: Finished Extra networking commands.. gitlab # [ 7.586835] systemd[1]: Reached target Network. gitlab # [ 7.588574] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 7.591031] dhcpcd[710]: dhcpcd-10.3.2 starting gitlab # [ 7.591878] systemd[1]: Starting Nginx Web Server... gitlab # [ 7.594394] dhcpcd[767]: dev: loaded udev gitlab # [ 7.595380] systemd[1]: Starting PostgreSQL Server... gitlab # [ 7.598608] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 7.750039] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 7.750681] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 7.603361] systemd[1]: Starting Permit User Sessions... gitlab # [ 7.620291] systemd[1]: Finished Permit User Sessions. gitlab # [ 7.623827] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 7.626779] systemd[1]: Started Getty on tty1. gitlab # [ 7.627508] systemd[1]: Reached target Login Prompts. gitlab # [ 7.841686] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 7.849469] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 7.850486] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 7.852155] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 7.853072] cfg80211: failed to load regulatory.db gitlab # [ 7.729473] nginx-pre-start[793]: nginx: the configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf syntax is ok gitlab # [ 7.731351] nginx-pre-start[793]: nginx: configuration file /nix/store/ahhadigcx1wjmb1izvs0lkn856wz7zi5-nginx.conf test is successful gitlab # [ 7.883189] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 7.736274] dhcpcd[767]: eth0: waiting for carrier gitlab # [ 7.738363] dhcpcd[767]: eth0: carrier acquired gitlab # [ 7.740124] systemd[1]: Started Nginx Web Server. gitlab # [ 7.741210] redis-server[789]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 7.745326] redis-server[789]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 7.746632] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 7.748084] postgresql-pre-start[794]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 7.749743] postgresql-pre-start[794]: This user must also own the server process. gitlab # [ 7.750965] postgresql-pre-start[794]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 7.752221] postgresql-pre-start[794]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 7.753626] postgresql-pre-start[794]: The default text search configuration will be set to "english". gitlab # [ 7.754934] postgresql-pre-start[794]: Data page checksums are enabled. gitlab # [ 7.755894] postgresql-pre-start[794]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 7.757329] postgresql-pre-start[794]: creating subdirectories ... ok gitlab # [ 7.758249] postgresql-pre-start[794]: selecting dynamic shared memory implementation ... posix gitlab # [ 7.759623] redis-server[789]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=789, just started gitlab # [ 7.761300] redis-server[789]: Configuration loaded gitlab # [ 7.762422] redis[789]: monotonic clock: POSIX clock_gettime gitlab # [ 7.763616] redis[789]: Running mode=standalone, port=0. gitlab # [ 7.764790] redis[789]: Server initialized gitlab # [ 7.765668] redis[789]: Ready to accept connections unix gitlab # [ 7.766708] redis[789]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 7.768679] dhcpcd[767]: DUID 00:01:00:01:32:46:e2:41:52:54:00:12:34:56 gitlab # [ 7.769619] dhcpcd[767]: eth0: IAID 00:12:34:56 gitlab # [ 7.770411] dhcpcd[767]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 7.778619] postgresql-pre-start[794]: selecting default "max_connections" ... 100 gitlab # [ 7.813413] postgresql-pre-start[794]: selecting default "shared_buffers" ... 128MB gitlab # [ 8.115637] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.141227] dhcpcd[767]: eth0: soliciting a DHCP lease gitlab # [ 8.295477] NET: Registered PF_PACKET protocol family gitlab # [ 8.155719] dhcpcd[767]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 8.159196] dhcpcd[767]: eth0: probing address 10.0.2.15/24 gitlab # [ 8.211298] postgresql-pre-start[794]: selecting default time zone ... UTC gitlab # [ 8.214179] postgresql-pre-start[794]: creating configuration files ... ok gitlab # [ 8.336965] gitlab-workhorse[825]: time="2026-09-23T19:17:21Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 8.340904] gitlab-workhorse[825]: time="2026-09-23T19:17:21Z" level=info msg="Using redis/go-redis" gitlab # [ 8.343959] gitlab-workhorse[825]: time="2026-09-23T19:17:21Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 8.383446] postgresql-pre-start[794]: running bootstrap script ... ok gitlab # [ 8.852697] postgresql-pre-start[794]: performing post-bootstrap initialization ... ok gitlab # [ 8.910184] dhcpcd[767]: eth0: soliciting an IPv6 router gitlab # [ 8.910979] dhcpcd[767]: eth0: Router Advertisement from fe80::2 gitlab # [ 8.912172] dhcpcd[767]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 8.913155] dhcpcd[767]: eth0: adding route to fec0::/64 gitlab # [ 8.915023] dhcpcd[767]: eth0: adding default route via fe80::2 gitlab # [ 9.163338] postgresql-pre-start[794]: syncing data to disk ... ok gitlab # [ 9.163688] postgresql-pre-start[794]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 9.165168] postgresql-pre-start[794]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 9.166271] postgresql-pre-start[794]: Success. You can now start the database server using: gitlab # [ 9.167445] postgresql-pre-start[794]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 9.299401] postgres[851]: [851] LOG: starting PostgreSQL 18.4 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 9.300934] postgres[851]: [851] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 9.301767] postgres[851]: [851] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 9.305391] postgres[851]: [851] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 9.320930] postgres[865]: [865] LOG: database system was shut down at 2026-09-23 19:17:22 GMT gitlab # [ 9.328863] postgres[851]: [851] LOG: database system is ready to accept connections gitlab # [ 9.334069] systemd[1]: Started PostgreSQL Server. gitlab # [ 9.336347] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 9.351649] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[855]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 9.354186] prszvjpp8qfjqrb5rqcxg3jlzg7rbbr1-gitlab-config[855]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 9.411500] systemd[1]: Finished gitlab-config.service. gitlab # [ 9.415599] systemd[1]: Started gitaly.service. gitlab # [ 9.514164] postgresql-setup-start[887]: CREATE DATABASE gitlab # [ 9.534225] postgresql-setup-start[904]: CREATE ROLE gitlab # [ 9.545798] postgresql-setup-start[906]: ALTER DATABASE gitlab # [ 9.565825] postgresql-setup-start[911]: CREATE ROLE gitlab # [ 9.567362] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 9.568780] systemd[1]: Reached target PostgreSQL. gitlab # [ 9.570766] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 9.572467] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 9.657338] gitlab-postgresql-start[927]: CREATE DATABASE gitlab # [ 9.689583] gitlab-postgresql-start[932]: CREATE EXTENSION gitlab # [ 9.773271] gitlab-postgresql-start[934]: CREATE EXTENSION gitlab # [ 9.775572] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 9.908349] gitaly[890]: time="2026-09-23T19:17:23.180Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/ymnlr20kjfmhkzg7i9hp25h3ii8qnn0d-gitaly.toml' and fix the errors that are printed." pid=890 gitlab # [ 9.909214] gitaly[890]: time="2026-09-23T19:17:23.180Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=890 gitlab # [ 9.910417] gitaly[890]: time="2026-09-23T19:17:23.180Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=890 gitlab # [ 9.911192] gitaly[890]: time="2026-09-23T19:17:23.180Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=890 gitlab # [ 9.911872] gitaly[890]: time="2026-09-23T19:17:23.180Z" level=info msg="Starting Gitaly" pid=890 version= gitlab # [ 10.100316] gitaly[890]: time="2026-09-23T19:17:23.373Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=181 pid=890 gitlab # [ 10.100778] gitaly[890]: time="2026-09-23T19:17:23.373Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=890 gitlab # [ 10.102140] gitaly[890]: time="2026-09-23T19:17:23.374Z" level=info msg="finished initializing command factory" duration_ms=1 pid=890 gitlab # [ 10.102834] gitaly[890]: time="2026-09-23T19:17:23.374Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-890/git-exec-3550909672.d/git pid=890 gitlab # [ 10.108988] gitaly[890]: time="2026-09-23T19:17:23.381Z" level=info msg="finished detecting git version" duration_ms=6 pid=890 gitlab # [ 10.109482] gitaly[890]: time="2026-09-23T19:17:23.381Z" level=info msg="using Git version" pid=890 version=2.53.g gitlab # [ 10.109934] gitaly[890]: time="2026-09-23T19:17:23.382Z" level=info msg="clearing disk cache object folder" pid=890 storage=default gitlab # [ 10.110602] gitaly[890]: time="2026-09-23T19:17:23.382Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2621467491 pid=890 storage=default gitlab # [ 10.111101] gitaly[890]: time="2026-09-23T19:17:23.382Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2621467491 pid=890 storage=default gitlab # [ 10.112352] gitaly[890]: time="2026-09-23T19:17:23.385Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2621467491 clear_duration_ms=0 pid=890 storage=default gitlab # [ 10.121155] gitaly[890]: time="2026-09-23T19:17:23.394Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=890 gitlab # [ 10.121584] gitaly[890]: time="2026-09-23T19:17:23.394Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=890 gitlab # [ 10.160288] gitaly[890]: time="2026-09-23T19:17:23.432Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=890 gitlab # [ 10.160733] gitaly[890]: time="2026-09-23T19:17:23.432Z" level=info msg="counting repositories" pid=890 gitlab # [ 10.161171] gitaly[890]: time="2026-09-23T19:17:23.432Z" level=info msg="finished tempdir cleaner walk" pid=890 storage=default time_ms=0 gitlab # [ 10.161940] gitaly[890]: time="2026-09-23T19:17:23.432Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=890 schema=unix gitlab # [ 10.162467] gitaly[890]: time="2026-09-23T19:17:23.432Z" level=info msg="starting to count repositories" pid=890 storage_path=/var/gitlab/state/repositories gitlab # [ 10.162942] gitaly[890]: time="2026-09-23T19:17:23.432Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=890 storage_path=/var/gitlab/state/repositories gitlab # [ 10.163496] gitaly[890]: time="2026-09-23T19:17:23.432Z" level=info msg="completed counting all repositories" counting_duration=0 pid=890 gitlab # [ 10.164111] gitaly[890]: time="2026-09-23T19:17:23.432Z" level=info msg="listening at address" address=/run/gitaly/gitaly-890/sock.d/intern pid=890 schema=unix gitlab # [ 10.175842] gitaly[890]: time="2026-09-23T19:17:23.448Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=890 gitlab # [ 10.176488] gitaly[890]: time="2026-09-23T19:17:23.448Z" level=info msg="Started Gitaly" duration_ms=267 pid=890 gitlab # [ 10.177111] gitaly[890]: time="2026-09-23T19:17:23.448Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=890 gitlab # [ 10.178995] gitaly[890]: time="2026-09-23T19:17:23.451Z" level=info msg="maintenance: daily scheduled" pid=890 scheduled="2026-09-24 12:00:00 +0000 UTC" gitlab # [ 11.103800] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[916]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.104546] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[916]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 11.143865] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[916]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.100740] gitaly[890]: time="2026-09-23T19:17:25.372Z" level=info msg="License database preloaded" duration_ms=2191 pid=890 gitlab # [ 13.491318] dhcpcd[767]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 13.492192] dhcpcd[767]: eth0: adding route to 10.0.2.0/24 gitlab # [ 13.493971] dhcpcd[767]: eth0: adding default route via 10.0.2.2 gitlab # [ 13.619244] systemd[1]: Started DHCP Client. gitlab # [ 13.622167] systemd[1]: Reached target Network is Online. gitlab # [ 41.431702] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[916]: W, [2026-09-23T19:17:54.703516 #916] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 43.672064] postgres[1316]: [1316] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 43.672308] postgres[1316]: [1316] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 43.673231] postgres[1316]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 43.674340] postgres[1316]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 43.675837] postgres[1316]: attidentity AS identity, gitlab # [ 43.676983] postgres[1316]: attgenerated as attgenerated gitlab # [ 43.678498] postgres[1316]: FROM pg_attribute a gitlab # [ 43.679966] postgres[1316]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 43.681470] postgres[1316]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 43.682572] postgres[1316]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 43.684083] postgres[1316]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 43.685200] postgres[1316]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 43.686333] postgres[1316]: ORDER BY a.attnum gitlab # [ 43.687814] postgres[1316]: gitlab # [ 47.062587] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1349]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 47.062827] j52kjsy7niw11krx8yhb7ipfwm07y66n-gitlab-db-config[1349]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 48.400649] gitlab-workhorse[825]: time="2026-09-23T19:18:01Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 58.409992] gitlab-workhorse[825]: time="2026-09-23T19:18:11Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 61.868688] postgres[1424]: [1424] ERROR: relation "table_without_models" does not exist gitlab # [ 61.869245] postgres[1424]: [1424] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 68.411236] gitlab-workhorse[825]: time="2026-09-23T19:18:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 73.914870] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 73.922090] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 73.925632] systemd[1]: Starting gitlab.service... gitlab # [ 74.289963] bundle[1591]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 74.290954] bundle[1591]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 74.291905] bundle[1591]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 76.957126] sidekiq-cluster[1590]: {"severity":"INFO","time":"2026-09-23T19:18:30.212Z","message":"Starting cluster with 1 processes"} gitlab # [ 77.081108] bundle[1591]: {"timestamp":"2026-09-23T19:18:30.333Z","pid":1591,"message":"* Enabling systemd notification integration"} gitlab # [ 77.081436] bundle[1591]: {"timestamp":"2026-09-23T19:18:30.353Z","pid":1591,"message":"Puma starting in cluster mode..."} gitlab # [ 77.083157] bundle[1591]: {"timestamp":"2026-09-23T19:18:30.353Z","pid":1591,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 77.083952] bundle[1591]: {"timestamp":"2026-09-23T19:18:30.353Z","pid":1591,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 77.085416] bundle[1591]: {"timestamp":"2026-09-23T19:18:30.353Z","pid":1591,"message":"* Min threads: 0"} gitlab # [ 77.086898] bundle[1591]: {"timestamp":"2026-09-23T19:18:30.353Z","pid":1591,"message":"* Max threads: 4"} gitlab # [ 77.088421] bundle[1591]: {"timestamp":"2026-09-23T19:18:30.353Z","pid":1591,"message":"* Environment: production"} gitlab # [ 77.089907] bundle[1591]: {"timestamp":"2026-09-23T19:18:30.353Z","pid":1591,"message":"* Master PID: 1591"} gitlab # [ 77.091457] bundle[1591]: {"timestamp":"2026-09-23T19:18:30.353Z","pid":1591,"message":"* Workers: 2"} gitlab # [ 77.092958] bundle[1591]: {"timestamp":"2026-09-23T19:18:30.353Z","pid":1591,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 77.094477] bundle[1591]: {"timestamp":"2026-09-23T19:18:30.353Z","pid":1591,"message":"* Preloading application"} gitlab # [ 78.412470] gitlab-workhorse[825]: time="2026-09-23T19:18:31Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 88.413766] gitlab-workhorse[825]: time="2026-09-23T19:18:41Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 98.415135] gitlab-workhorse[825]: time="2026-09-23T19:18:51Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 99.360887] bundle[1591]: W, [2026-09-23T19:18:52.633030 #1591] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 101.690112] sidekiq-cluster[1621]: 2026-09-23T19:18:54.961Z pid=1621 tid=wl9 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1621, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>432676864} gitlab # [ 101.858087] sidekiq-cluster[1621]: W, [2026-09-23T19:18:55.130649 #1621] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 101.897147] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:55.169Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 103.826812] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.098Z","message":"GitLab reliable fetch activated!"} gitlab # [ 103.830566] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.098Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 103.856463] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.129Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 103.861715] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.133Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 103.866979] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.137Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 103.871950] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.141Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 103.877489] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.145Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 103.882421] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.149Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 103.887473] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.152Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 103.892133] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.155Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 103.896906] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.158Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 103.900864] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.161Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 103.905069] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.164Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 103.909096] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.167Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 103.912784] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.170Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 103.916846] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.173Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 103.919069] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.175Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 103.921151] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.177Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 103.923090] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.179Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 103.925073] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.181Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 103.927238] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.183Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 103.930442] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.185Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 103.932698] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.187Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 103.935020] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.188Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 103.937098] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.190Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 103.939313] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.192Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 103.941417] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.193Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 103.943398] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.195Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 103.945327] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.197Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 103.947412] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.199Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 103.949428] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.201Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 103.951448] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.203Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 103.953345] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.207Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 103.955591] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.209Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 103.957558] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.211Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 103.959694] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.213Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 103.961826] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.214Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 103.963936] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.216Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 103.965990] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.217Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 103.967989] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.220Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 103.970077] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.221Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 103.971991] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.223Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 103.976106] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.224Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 103.978198] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.226Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 103.980295] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.228Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 103.982341] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.229Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 103.984416] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.231Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 103.986707] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.233Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 103.988822] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.235Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 103.990886] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.236Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 103.992894] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.238Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 103.994981] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.240Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 103.997106] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.242Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 103.999373] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.244Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 104.001527] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.253Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 104.004064] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.255Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 104.004520] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.257Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 104.005074] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.260Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 104.005505] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.262Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 104.005991] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.266Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 104.007143] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.270Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 104.007598] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.273Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 104.008090] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.275Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 104.008595] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.281Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 104.011079] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.283Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 104.013530] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.286Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 104.015472] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.288Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 104.049737] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.322Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 104.053137] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.326Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 104.056594] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.329Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 104.060277] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.333Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 104.064462] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.337Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 104.069627] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.342Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 104.074088] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.346Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 104.077560] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.350Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 104.081123] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.353Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 104.085359] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.358Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 104.087468] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.360Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 104.089606] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.362Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 104.091155] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.364Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 104.093584] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.366Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 104.095170] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.368Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 104.097108] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.369Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 104.099519] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.372Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 104.102466] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.375Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 104.104137] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.377Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 104.106899] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.379Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 104.108528] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.381Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 104.111274] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.384Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 104.112926] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.385Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 104.115414] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.388Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 104.117129] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.390Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 104.119078] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.391Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 104.121085] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.393Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 104.122957] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.395Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 104.125328] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.398Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 104.127780] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.400Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 104.130583] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.403Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 104.132197] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.405Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 104.134688] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.407Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 104.144133] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.416Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 104.145598] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.418Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 104.147506] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.420Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 104.150595] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.423Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 104.152674] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.425Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 104.155026] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.427Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 104.156990] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.429Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 104.159085] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.431Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 104.160892] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.433Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 104.163875] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.436Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 104.165484] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:18:57.438Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 108.425386] gitlab-workhorse[825]: time="2026-09-23T19:19:01Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 118.435811] gitlab-workhorse[825]: time="2026-09-23T19:19:11Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 128.134101] postgres[2059]: [2059] ERROR: relation "table_without_models" does not exist gitlab # [ 128.134889] postgres[2059]: [2059] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 128.438397] gitlab-workhorse[825]: time="2026-09-23T19:19:21Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 129.532349] postgres[2070]: [2070] ERROR: relation "table_without_models" does not exist gitlab # [ 129.532958] postgres[2070]: [2070] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 130.711854] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:23.984Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 130.716068] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:23.988Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 130.716182] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:23.988Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 130.716954] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:23.988Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 130.730360] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:24.003Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 132.018368] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:25.290Z","message":"Cleaning working queues"} gitlab # [ 132.023602] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:25.296Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"031ef44a8a9584decc722a59","created_at":"2026-09-23T19:18:26.819Z","trace_propagation_headers":{"sentry-trace":"880184d983e942c4a64c89f7b9476480-70add91cf0dd4666","baggage":"sentry-trace_id=880184d983e942c4a64c89f7b9476480,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"b25b75d03a60d93d3d62d2183769318b","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-23T19:18:26.826Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-031ef44a8a9584decc722a59: start","job_status":"start","queue_duration_s":58.469639,"scheduling_latency_s":58.469777} gitlab # [ 132.211967] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:25.484Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"031ef44a8a9584decc722a59","created_at":"2026-09-23T19:18:26.819Z","trace_propagation_headers":{"sentry-trace":"880184d983e942c4a64c89f7b9476480-70add91cf0dd4666","baggage":"sentry-trace_id=880184d983e942c4a64c89f7b9476480,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"b25b75d03a60d93d3d62d2183769318b","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-23T19:18:26.826Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-031ef44a8a9584decc722a59: done: 0.188707 sec","job_status":"done","queue_duration_s":58.469639,"scheduling_latency_s":58.469777,"redis_calls":31,"redis_duration_s":0.008045,"redis_read_bytes":284,"redis_write_bytes":3897,"redis_feature_flag_calls":20,"redis_feature_flag_duration_s":0.004497,"redis_feature_flag_read_bytes":274,"redis_feature_flag_write_bytes":2650,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002074,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001474,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":16,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":16,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.07384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09550710999999999,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.188707,"completed_at":"2026-09-23T19:19:25.484Z","load_balancing_strategy":"primary","db_duration_s":0.063789,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 132.299038] bundle[1591]: {"timestamp":"2026-09-23T19:19:25.545Z","pid":1591,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 132.301964] bundle[1591]: {"timestamp":"2026-09-23T19:19:25.545Z","pid":1591,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 132.302618] bundle[1591]: {"timestamp":"2026-09-23T19:19:25.545Z","pid":1591,"message":"! #\u003cThread:0x000070a2123a4588 /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/0pw9q4cx4pc2icd2n81q1c3zi7y56z3a-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 132.303485] bundle[1591]: {"timestamp":"2026-09-23T19:19:25.545Z","pid":1591,"message":"! #\u003cThread:0x000070a2125ebc38 /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/zyib4fdxd0zl63b2nm6163j5jb69nbrk-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 132.304961] bundle[1591]: {"timestamp":"2026-09-23T19:19:25.545Z","pid":1591,"message":"Use Ctrl-C to stop"} gitlab # [ 132.306109] bundle[1591]: {"timestamp":"2026-09-23T19:19:25.545Z","pid":1591,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 132.592606] bundle[1591]: {"timestamp":"2026-09-23T19:19:25.864Z","pid":1591,"message":"- Worker 0 (PID: 2162) booted in 0.29s, phase: 0"} gitlab # [ 132.603525] bundle[1591]: {"timestamp":"2026-09-23T19:19:25.876Z","pid":1591,"message":"- Worker 1 (PID: 2166) booted in 0.29s, phase: 0"} gitlab # [ 132.611546] systemd[1]: Started gitlab.service. gitlab # [ 132.612384] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 132.616267] systemd[1]: Starting setup-gitlab.service... gitlab # [ 132.639331] setup-gitlab-start[2199]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 134.981790] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:19:28 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 2317 gitlab # [ 134.990100] setup-gitlab-start[2199]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 134.990902] setup-gitlab-start[2199]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 135.020048] runuser[2217]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 135.404904] setup-gitlab-start[2218]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 135.405422] setup-gitlab-start[2218]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 135.406540] setup-gitlab-start[2218]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 135.972881] setup-gitlab-start[2218]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 135.973326] setup-gitlab-start[2218]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 135.974876] setup-gitlab-start[2218]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 146.514753] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:39.786Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"bb05090a95d6a23d28156334","created_at":"2026-09-23T19:19:39.753Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"370b8a18b31617a391abdde287177534","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:39.783Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-bb05090a95d6a23d28156334: start","job_status":"start","queue_duration_s":0.002763,"scheduling_latency_s":0.002813} gitlab # [ 147.148163] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.420Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"bb05090a95d6a23d28156334","created_at":"2026-09-23T19:19:39.753Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"370b8a18b31617a391abdde287177534","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:39.783Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-bb05090a95d6a23d28156334: done: 0.6345 sec","job_status":"done","queue_duration_s":0.002763,"scheduling_latency_s":0.002813,"redis_calls":565,"redis_duration_s":0.31371299999999996,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00158,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002425,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.309708,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.289257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000320432,"duration_s":0.6345,"completed_at":"2026-09-23T19:19:40.420Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.186751] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.459Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"84c76ff8ba430abee475abbc","created_at":"2026-09-23T19:19:39.842Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ccc26b82c3eab8c813048f9210ece13","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:39.858Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-84c76ff8ba430abee475abbc: start","job_status":"start","queue_duration_s":0.601112,"scheduling_latency_s":0.601135} gitlab # [ 147.251171] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.523Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"84c76ff8ba430abee475abbc","created_at":"2026-09-23T19:19:39.842Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ccc26b82c3eab8c813048f9210ece13","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:39.858Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-84c76ff8ba430abee475abbc: done: 0.063875 sec","job_status":"done","queue_duration_s":0.601112,"scheduling_latency_s":0.601135,"redis_calls":20,"redis_duration_s":0.005872,"redis_read_bytes":20,"redis_write_bytes":3248,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002335,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1346,"redis_queues_calls":2,"redis_queues_duration_s":0.000414,"redis_queues_read_bytes":3,"redis_queues_write_bytes":818,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00212,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":734,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":350,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000485257,"duration_s":0.063875,"completed_at":"2026-09-23T19:19:40.523Z","load_balancing_strategy":"primary","db_duration_s":0.002087,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.292287] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.564Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"9a5a3403c879ed418c33073d","created_at":"2026-09-23T19:19:39.875Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7eeaaf431d476f8dee8fec34582c28c9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:39.888Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-9a5a3403c879ed418c33073d: start","job_status":"start","queue_duration_s":0.67588,"scheduling_latency_s":0.675913} gitlab # [ 147.322580] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.595Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 147.335597] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.607Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"9a5a3403c879ed418c33073d","created_at":"2026-09-23T19:19:39.875Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7eeaaf431d476f8dee8fec34582c28c9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:39.888Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-9a5a3403c879ed418c33073d: done: 0.04242 sec","job_status":"done","queue_duration_s":0.67588,"scheduling_latency_s":0.675913,"redis_calls":10,"redis_duration_s":0.0032790000000000002,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00063,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001891,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000758,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000559569,"duration_s":0.04242,"completed_at":"2026-09-23T19:19:40.607Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000793,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.337666] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.610Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"d8111e3c4ce27e91e7177856","created_at":"2026-09-23T19:19:39.898Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e98d5126fda83eb23eff049443c59ddf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:39.907Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-d8111e3c4ce27e91e7177856: start","job_status":"start","queue_duration_s":0.702354,"scheduling_latency_s":0.702382} gitlab # [ 147.367317] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.639Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"d8111e3c4ce27e91e7177856","created_at":"2026-09-23T19:19:39.898Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e98d5126fda83eb23eff049443c59ddf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:39.907Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-d8111e3c4ce27e91e7177856: done: 0.02966 sec","job_status":"done","queue_duration_s":0.702354,"scheduling_latency_s":0.702382,"redis_calls":19,"redis_duration_s":0.004443,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001045,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000324,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002325,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000749,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017869,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000265397,"duration_s":0.02966,"completed_at":"2026-09-23T19:19:40.639Z","load_balancing_strategy":"primary","db_duration_s":0.002192,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.384099] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.640Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"9c008c8a1643ab8b29df6a32","created_at":"2026-09-23T19:19:39.917Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c8cfb84cfd2374c3f10e28b6c505f6a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:39.928Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-9c008c8a1643ab8b29df6a32: start","job_status":"start","queue_duration_s":0.712597,"scheduling_latency_s":0.712611} gitlab # [ 147.398081] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.659Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"9c008c8a1643ab8b29df6a32","created_at":"2026-09-23T19:19:39.917Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c8cfb84cfd2374c3f10e28b6c505f6a","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:39.928Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-9c008c8a1643ab8b29df6a32: done: 0.018798 sec","job_status":"done","queue_duration_s":0.712597,"scheduling_latency_s":0.712611,"redis_calls":10,"redis_duration_s":0.0014680000000000001,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000251,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000457,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000209523,"duration_s":0.018798,"completed_at":"2026-09-23T19:19:40.659Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005329,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.444079] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.660Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"50f57edf2f4a20328c5dc9b2","created_at":"2026-09-23T19:19:39.942Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63a8bcfc2bbc835e58f4506225a5ba93","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:39.953Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-50f57edf2f4a20328c5dc9b2: start","job_status":"start","queue_duration_s":0.707306,"scheduling_latency_s":0.707319} gitlab # [ 147.446041] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.675Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"50f57edf2f4a20328c5dc9b2","created_at":"2026-09-23T19:19:39.942Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63a8bcfc2bbc835e58f4506225a5ba93","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:39.953Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-50f57edf2f4a20328c5dc9b2: done: 0.015167 sec","job_status":"done","queue_duration_s":0.707306,"scheduling_latency_s":0.707319,"redis_calls":12,"redis_duration_s":0.0015990000000000002,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000272,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000738,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000589,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000334121,"exclusive_lock_hold_duration_s":0.002211455,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.015167,"completed_at":"2026-09-23T19:19:40.675Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002042,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.491747] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.676Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e99568dc909fd840047ee89d","created_at":"2026-09-23T19:19:39.962Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60721a4eadaa9eea230cc0e5b93aee01","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:39.972Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e99568dc909fd840047ee89d: start","job_status":"start","queue_duration_s":0.703938,"scheduling_latency_s":0.703951} gitlab # [ 147.532584] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.684Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 147.533202] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.688Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e99568dc909fd840047ee89d","created_at":"2026-09-23T19:19:39.962Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60721a4eadaa9eea230cc0e5b93aee01","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:39.972Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e99568dc909fd840047ee89d: done: 0.012148 sec","job_status":"done","queue_duration_s":0.703938,"scheduling_latency_s":0.703951,"redis_calls":10,"redis_duration_s":0.00164,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000362,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000891,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000387,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000245283,"duration_s":0.012148,"completed_at":"2026-09-23T19:19:40.688Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000253,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.537052] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.689Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"fcb50c0d72cd759804e56585","created_at":"2026-09-23T19:19:40.001Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd54df978c34e0bb6395cdba5ce35158","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.009Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-fcb50c0d72cd759804e56585: start","job_status":"start","queue_duration_s":0.680752,"scheduling_latency_s":0.680765} gitlab # [ 147.549997] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.704Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"fcb50c0d72cd759804e56585","created_at":"2026-09-23T19:19:40.001Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd54df978c34e0bb6395cdba5ce35158","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.009Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-fcb50c0d72cd759804e56585: done: 0.014941 sec","job_status":"done","queue_duration_s":0.680752,"scheduling_latency_s":0.680765,"redis_calls":12,"redis_duration_s":0.002545,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000884,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001279,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025981,"duration_s":0.014941,"completed_at":"2026-09-23T19:19:40.704Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000752,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.601763] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.705Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"380520dd9d01912ca5a10dfa","created_at":"2026-09-23T19:19:40.019Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7909c21edd7b11e6b9022200cb5e2898","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-23T19:19:40.026Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-380520dd9d01912ca5a10dfa: start","job_status":"start","queue_duration_s":0.679164,"scheduling_latency_s":0.679177} gitlab # [ 147.602600] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.713Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 147.603096] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.723Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"380520dd9d01912ca5a10dfa","created_at":"2026-09-23T19:19:40.019Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7909c21edd7b11e6b9022200cb5e2898","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-23T19:19:40.026Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-380520dd9d01912ca5a10dfa: done: 0.017604 sec","job_status":"done","queue_duration_s":0.679164,"scheduling_latency_s":0.679177,"redis_calls":10,"redis_duration_s":0.002313,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000441,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001435,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000279645,"duration_s":0.017604,"completed_at":"2026-09-23T19:19:40.723Z","load_balancing_strategy":"primary","db_duration_s":0.000414,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.653750] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.726Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"55f787d672fa2df788f62602","created_at":"2026-09-23T19:19:40.037Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"185eb22fbbc5ee2591a750611c17753d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-23T19:19:40.054Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-55f787d672fa2df788f62602: start","job_status":"start","queue_duration_s":0.671859,"scheduling_latency_s":0.671895} gitlab # [ 147.654445] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.753Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"55f787d672fa2df788f62602","created_at":"2026-09-23T19:19:40.037Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"185eb22fbbc5ee2591a750611c17753d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-23T19:19:40.054Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-55f787d672fa2df788f62602: done: 0.027983 sec","job_status":"done","queue_duration_s":0.671859,"scheduling_latency_s":0.671895,"redis_calls":10,"redis_duration_s":0.00202,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000505,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000378,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208685,"duration_s":0.027983,"completed_at":"2026-09-23T19:19:40.753Z","load_balancing_strategy":"primary","db_duration_s":0.002565,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.695138] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.755Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"b8ca8015c250a2a545c99c54","created_at":"2026-09-23T19:19:40.072Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"44f472388184b0db5682de6eac1c9863","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.114Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-b8ca8015c250a2a545c99c54: start","job_status":"start","queue_duration_s":0.640984,"scheduling_latency_s":0.640998} gitlab # [ 147.698911] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.774Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"b8ca8015c250a2a545c99c54","created_at":"2026-09-23T19:19:40.072Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"44f472388184b0db5682de6eac1c9863","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.114Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-b8ca8015c250a2a545c99c54: done: 0.01939 sec","job_status":"done","queue_duration_s":0.640984,"scheduling_latency_s":0.640998,"redis_calls":13,"redis_duration_s":0.001743,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000577,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0058272770000000005,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.01939,"completed_at":"2026-09-23T19:19:40.774Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001539,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 147.737792] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.775Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"196196bb67df3da7ab2213de","created_at":"2026-09-23T19:19:40.123Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cad1cc8f7556c21d22a69fb421bd144","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.131Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-196196bb67df3da7ab2213de: start","job_status":"start","queue_duration_s":0.643798,"scheduling_latency_s":0.643812} gitlab # [ 147.751060] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.782Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 147.754256] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.787Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"196196bb67df3da7ab2213de","created_at":"2026-09-23T19:19:40.123Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cad1cc8f7556c21d22a69fb421bd144","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.131Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-196196bb67df3da7ab2213de: done: 0.011887 sec","job_status":"done","queue_duration_s":0.643798,"scheduling_latency_s":0.643812,"redis_calls":10,"redis_duration_s":0.00129,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000244,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006618,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000200584,"duration_s":0.011887,"completed_at":"2026-09-23T19:19:40.787Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000285,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.807295] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.788Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b07e133bf8ed0759b5e5b630","created_at":"2026-09-23T19:19:40.138Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d36ea21a04e4ac15a1da4406182de9ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.145Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b07e133bf8ed0759b5e5b630: start","job_status":"start","queue_duration_s":0.642554,"scheduling_latency_s":0.642568} gitlab # [ 147.807474] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.795Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 147.808913] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b07e133bf8ed0759b5e5b630","created_at":"2026-09-23T19:19:40.138Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d36ea21a04e4ac15a1da4406182de9ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.145Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b07e133bf8ed0759b5e5b630: done: 0.011255 sec","job_status":"done","queue_duration_s":0.642554,"scheduling_latency_s":0.642568,"redis_calls":10,"redis_duration_s":0.001196,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000239,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000712,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000245,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006429,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000196673,"duration_s":0.011255,"completed_at":"2026-09-23T19:19:40.799Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000228,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.851892] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.800Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"77d1bd9e7b3c27bfda41541f","created_at":"2026-09-23T19:19:40.157Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31c4ac80dc0deacc7c9b1daf0693a18e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.165Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-77d1bd9e7b3c27bfda41541f: start","job_status":"start","queue_duration_s":0.635434,"scheduling_latency_s":0.635447} gitlab # [ 147.858770] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.842Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"77d1bd9e7b3c27bfda41541f","created_at":"2026-09-23T19:19:40.157Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"31c4ac80dc0deacc7c9b1daf0693a18e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.165Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-77d1bd9e7b3c27bfda41541f: done: 0.041739 sec","job_status":"done","queue_duration_s":0.635434,"scheduling_latency_s":0.635447,"redis_calls":14,"redis_duration_s":0.0019190000000000001,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000567,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000595,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000311213,"exclusive_lock_hold_duration_s":0.027908296,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.041739,"completed_at":"2026-09-23T19:19:40.842Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007811,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.899072] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.843Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"0fd5d615398c501b01962b51","created_at":"2026-09-23T19:19:40.174Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3a42a0d7f9788b48ae1a9b3724def41","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.181Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-0fd5d615398c501b01962b51: start","job_status":"start","queue_duration_s":0.662403,"scheduling_latency_s":0.662419} gitlab # [ 147.937766] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.967Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"0fd5d615398c501b01962b51","created_at":"2026-09-23T19:19:40.174Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3a42a0d7f9788b48ae1a9b3724def41","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.181Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-0fd5d615398c501b01962b51: done: 0.124302 sec","job_status":"done","queue_duration_s":0.662403,"scheduling_latency_s":0.662419,"redis_calls":12,"redis_duration_s":0.0017389999999999999,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000833,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000664,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.100857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000312889,"exclusive_lock_hold_duration_s":0.11057271299999999,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.124302,"completed_at":"2026-09-23T19:19:40.967Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027215,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 147.950602] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.969Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"4e9f1d91f265850f464486d9","created_at":"2026-09-23T19:19:40.198Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e1c58b28d976879ec355c5e553b6b82","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-23T19:19:40.204Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-4e9f1d91f265850f464486d9: start","job_status":"start","queue_duration_s":0.764271,"scheduling_latency_s":0.764285} gitlab # [ 147.962402] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.976Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 147.965928] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.981Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"4e9f1d91f265850f464486d9","created_at":"2026-09-23T19:19:40.198Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e1c58b28d976879ec355c5e553b6b82","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-23T19:19:40.204Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-4e9f1d91f265850f464486d9: done: 0.012148 sec","job_status":"done","queue_duration_s":0.764271,"scheduling_latency_s":0.764285,"redis_calls":10,"redis_duration_s":0.001246,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000247,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000731,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000197511,"duration_s":0.012148,"completed_at":"2026-09-23T19:19:40.981Z","load_balancing_strategy":"primary","db_duration_s":0.000314,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.023151] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:40.982Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ee9ad538926931062610cbb4","created_at":"2026-09-23T19:19:40.214Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.220Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ee9ad538926931062610cbb4: start","job_status":"start","queue_duration_s":0.761331,"scheduling_latency_s":0.761343} gitlab # [ 148.023808] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.045Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ee9ad538926931062610cbb4","created_at":"2026-09-23T19:19:40.214Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.220Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ee9ad538926931062610cbb4: done: 0.06361 sec","job_status":"done","queue_duration_s":0.761331,"scheduling_latency_s":0.761343,"redis_calls":71,"redis_duration_s":0.009495,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.00125,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000544,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.007222,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000215111,"duration_s":0.06361,"completed_at":"2026-09-23T19:19:41.045Z","load_balancing_strategy":"primary","db_duration_s":0.001023,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.052743] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.046Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"12b483dcc54b39a1fa15ae5a","created_at":"2026-09-23T19:19:40.229Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b8e7a6aa9e92d3d94c3fa5843a75ab2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.235Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-12b483dcc54b39a1fa15ae5a: start","job_status":"start","queue_duration_s":0.811287,"scheduling_latency_s":0.811301} gitlab # [ 148.093510] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.059Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"12b483dcc54b39a1fa15ae5a","created_at":"2026-09-23T19:19:40.229Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6b8e7a6aa9e92d3d94c3fa5843a75ab2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.235Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-12b483dcc54b39a1fa15ae5a: done: 0.013115 sec","job_status":"done","queue_duration_s":0.811287,"scheduling_latency_s":0.811301,"redis_calls":10,"redis_duration_s":0.001289,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000795,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000245,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000196673,"duration_s":0.013115,"completed_at":"2026-09-23T19:19:41.059Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000361,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.138252] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.060Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d1ccabd17a3b07bc90e21a23","created_at":"2026-09-23T19:19:40.248Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d66f589de243afddc0885e71bb19e499","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.254Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d1ccabd17a3b07bc90e21a23: start","job_status":"start","queue_duration_s":0.806055,"scheduling_latency_s":0.806067} gitlab # [ 148.138934] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.099Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d1ccabd17a3b07bc90e21a23","created_at":"2026-09-23T19:19:40.248Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d66f589de243afddc0885e71bb19e499","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.254Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d1ccabd17a3b07bc90e21a23: done: 0.038268 sec","job_status":"done","queue_duration_s":0.806055,"scheduling_latency_s":0.806067,"redis_calls":16,"redis_duration_s":0.003322,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002002,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000959,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000361,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000386641,"duration_s":0.038268,"completed_at":"2026-09-23T19:19:41.098Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00738,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.178715] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.101Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"398b330efaa022ddd160c5be","created_at":"2026-09-23T19:19:40.261Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d1c1a4d87041d3c089c7021fcc3b462","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.267Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-398b330efaa022ddd160c5be: start","job_status":"start","queue_duration_s":0.834173,"scheduling_latency_s":0.834214} gitlab # [ 148.179609] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.130Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"398b330efaa022ddd160c5be","created_at":"2026-09-23T19:19:40.261Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d1c1a4d87041d3c089c7021fcc3b462","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.267Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-398b330efaa022ddd160c5be: done: 0.028972 sec","job_status":"done","queue_duration_s":0.834173,"scheduling_latency_s":0.834214,"redis_calls":12,"redis_duration_s":0.00263,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000541,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001212,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000877,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000588343,"exclusive_lock_hold_duration_s":0.007124649,"duration_s":0.028972,"completed_at":"2026-09-23T19:19:41.130Z","load_balancing_strategy":"primary","db_duration_s":0.003196,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.200433] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.131Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7f83af24e9d59871e5271028","created_at":"2026-09-23T19:19:40.278Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06413564990439e884ba15f6349804f7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.283Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7f83af24e9d59871e5271028: start","job_status":"start","queue_duration_s":0.848162,"scheduling_latency_s":0.848178} gitlab # [ 148.247989] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.155Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"7f83af24e9d59871e5271028","created_at":"2026-09-23T19:19:40.278Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06413564990439e884ba15f6349804f7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.283Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-7f83af24e9d59871e5271028: done: 0.023943 sec","job_status":"done","queue_duration_s":0.848162,"scheduling_latency_s":0.848178,"redis_calls":12,"redis_duration_s":0.001558,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000247,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000734,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000577,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012386,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0003344,"exclusive_lock_hold_duration_s":0.008256078,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.023943,"completed_at":"2026-09-23T19:19:41.155Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.253177] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.156Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"287db55b28f6ab6b23534766","created_at":"2026-09-23T19:19:40.289Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6229d8cb0af6530a477f6633893c3814","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.294Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-287db55b28f6ab6b23534766: start","job_status":"start","queue_duration_s":0.862468,"scheduling_latency_s":0.862483} gitlab # [ 148.265691] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.177Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"287db55b28f6ab6b23534766","created_at":"2026-09-23T19:19:40.289Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6229d8cb0af6530a477f6633893c3814","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.294Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-287db55b28f6ab6b23534766: done: 0.02125 sec","job_status":"done","queue_duration_s":0.862468,"scheduling_latency_s":0.862483,"redis_calls":10,"redis_duration_s":0.001275,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000251,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000274,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002112,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.02125,"completed_at":"2026-09-23T19:19:41.177Z","load_balancing_strategy":"primary","db_duration_s":0.006997,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.314332] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.178Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"66971427bfc20af87b21b408","created_at":"2026-09-23T19:19:40.500Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-c3e50be0f3d54727","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"5ccc26b82c3eab8c813048f9210ece13","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.512Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-66971427bfc20af87b21b408: start","job_status":"start","queue_duration_s":0.665817,"scheduling_latency_s":0.66583} gitlab # [ 148.315052] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"66971427bfc20af87b21b408","created_at":"2026-09-23T19:19:40.500Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-c3e50be0f3d54727","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"5ccc26b82c3eab8c813048f9210ece13","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.512Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-66971427bfc20af87b21b408: done: 0.019904 sec","job_status":"done","queue_duration_s":0.665817,"scheduling_latency_s":0.66583,"redis_calls":13,"redis_duration_s":0.001874,"redis_read_bytes":12,"redis_write_bytes":1665,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000242,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":519,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":801,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012896,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00020701,"duration_s":0.019904,"completed_at":"2026-09-23T19:19:41.198Z","load_balancing_strategy":"primary","db_duration_s":0.002314,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.358605] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.199Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3cede5b2c91edebff1978309","created_at":"2026-09-23T19:19:40.626Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-f740f068691c485d","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e98d5126fda83eb23eff049443c59ddf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.634Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3cede5b2c91edebff1978309: start","job_status":"start","queue_duration_s":0.565247,"scheduling_latency_s":0.565259} gitlab # [ 148.360584] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.209Z","message":"No existing merge request to be cleaned up."} gitlab # [ 148.361883] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.216Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3cede5b2c91edebff1978309","created_at":"2026-09-23T19:19:40.626Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-f740f068691c485d","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e98d5126fda83eb23eff049443c59ddf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.634Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3cede5b2c91edebff1978309: done: 0.017144 sec","job_status":"done","queue_duration_s":0.565247,"scheduling_latency_s":0.565259,"redis_calls":13,"redis_duration_s":0.0016669999999999999,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000269,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000646,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000752,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000200864,"duration_s":0.017144,"completed_at":"2026-09-23T19:19:41.216Z","load_balancing_strategy":"primary","db_duration_s":0.001867,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.419656] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.217Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a1ba5bad057a6c794ff4ad95","created_at":"2026-09-23T19:19:40.626Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-f740f068691c485d","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e98d5126fda83eb23eff049443c59ddf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.634Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a1ba5bad057a6c794ff4ad95: start","job_status":"start","queue_duration_s":0.583236,"scheduling_latency_s":0.583248} gitlab # [ 148.420454] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.224Z","message":"No existing merge request to be cleaned up."} gitlab # [ 148.420950] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.228Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a1ba5bad057a6c794ff4ad95","created_at":"2026-09-23T19:19:40.626Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-f740f068691c485d","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e98d5126fda83eb23eff049443c59ddf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.634Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a1ba5bad057a6c794ff4ad95: done: 0.011231 sec","job_status":"done","queue_duration_s":0.583236,"scheduling_latency_s":0.583248,"redis_calls":10,"redis_duration_s":0.00131,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000771,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007527,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011231,"completed_at":"2026-09-23T19:19:41.228Z","load_balancing_strategy":"primary","db_duration_s":0.000888,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.459656] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.229Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"03fd04517be3be328fa88479","created_at":"2026-09-23T19:19:40.626Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-f740f068691c485d","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e98d5126fda83eb23eff049443c59ddf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.634Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-03fd04517be3be328fa88479: start","job_status":"start","queue_duration_s":0.595288,"scheduling_latency_s":0.595301} gitlab # [ 148.460503] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.236Z","message":"No existing merge request to be cleaned up."} gitlab # [ 148.460882] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.240Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"03fd04517be3be328fa88479","created_at":"2026-09-23T19:19:40.626Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-f740f068691c485d","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e98d5126fda83eb23eff049443c59ddf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.634Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-03fd04517be3be328fa88479: done: 0.011121 sec","job_status":"done","queue_duration_s":0.595288,"scheduling_latency_s":0.595301,"redis_calls":10,"redis_duration_s":0.0013189999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000772,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011121,"completed_at":"2026-09-23T19:19:41.240Z","load_balancing_strategy":"primary","db_duration_s":0.000898,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.503764] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.241Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b195e135384a81e22915ae01","created_at":"2026-09-23T19:19:40.626Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-f740f068691c485d","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e98d5126fda83eb23eff049443c59ddf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.634Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b195e135384a81e22915ae01: start","job_status":"start","queue_duration_s":0.607406,"scheduling_latency_s":0.60742} gitlab # [ 148.504636] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.248Z","message":"No existing merge request to be cleaned up."} gitlab # [ 148.505698] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.252Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b195e135384a81e22915ae01","created_at":"2026-09-23T19:19:40.626Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-f740f068691c485d","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e98d5126fda83eb23eff049443c59ddf","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:40.634Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b195e135384a81e22915ae01: done: 0.01098 sec","job_status":"done","queue_duration_s":0.607406,"scheduling_latency_s":0.60742,"redis_calls":10,"redis_duration_s":0.001267,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000745,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01098,"completed_at":"2026-09-23T19:19:41.252Z","load_balancing_strategy":"primary","db_duration_s":0.0009,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.564108] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3e36ab5d4fdca6d17d50081","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3e36ab5d4fdca6d17d50081: start","job_status":"start","queue_duration_s":0.21423,"scheduling_latency_s":0.214242} gitlab # [ 148.564910] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3e36ab5d4fdca6d17d50081","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3e36ab5d4fdca6d17d50081: done: 0.014836 sec","job_status":"done","queue_duration_s":0.21423,"scheduling_latency_s":0.214242,"redis_calls":16,"redis_duration_s":0.0021279999999999997,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000259,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001169,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00828,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000201143,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014836,"completed_at":"2026-09-23T19:19:41.268Z","load_balancing_strategy":"primary","db_duration_s":0.000231,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.585361] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2792cead81eb7a00cebf57ab","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2792cead81eb7a00cebf57ab: start","job_status":"start","queue_duration_s":0.229909,"scheduling_latency_s":0.229921} gitlab # [ 148.596852] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.279Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2792cead81eb7a00cebf57ab","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2792cead81eb7a00cebf57ab: done: 0.009635 sec","job_status":"done","queue_duration_s":0.229909,"scheduling_latency_s":0.229921,"redis_calls":13,"redis_duration_s":0.001689,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00067,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006146,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009635,"completed_at":"2026-09-23T19:19:41.279Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.642685] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.280Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efbfd14df18c357923bb7662","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efbfd14df18c357923bb7662: start","job_status":"start","queue_duration_s":0.240514,"scheduling_latency_s":0.240527} gitlab # [ 148.643595] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.289Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efbfd14df18c357923bb7662","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efbfd14df18c357923bb7662: done: 0.009569 sec","job_status":"done","queue_duration_s":0.240514,"scheduling_latency_s":0.240527,"redis_calls":13,"redis_duration_s":0.001559,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009569,"completed_at":"2026-09-23T19:19:41.289Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.678770] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"481359337dbc7392d66d93d1","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-481359337dbc7392d66d93d1: start","job_status":"start","queue_duration_s":0.251088,"scheduling_latency_s":0.251101} gitlab # [ 148.689709] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"481359337dbc7392d66d93d1","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-481359337dbc7392d66d93d1: done: 0.009635 sec","job_status":"done","queue_duration_s":0.251088,"scheduling_latency_s":0.251101,"redis_calls":13,"redis_duration_s":0.0015580000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009635,"completed_at":"2026-09-23T19:19:41.300Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.729684] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.301Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb49ca5c7db6ce61c12c87a5","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb49ca5c7db6ce61c12c87a5: start","job_status":"start","queue_duration_s":0.261804,"scheduling_latency_s":0.261817} gitlab # [ 148.735210] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.311Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb49ca5c7db6ce61c12c87a5","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb49ca5c7db6ce61c12c87a5: done: 0.009728 sec","job_status":"done","queue_duration_s":0.261804,"scheduling_latency_s":0.261817,"redis_calls":13,"redis_duration_s":0.001513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009728,"completed_at":"2026-09-23T19:19:41.310Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.774068] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.312Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53d6f0cf08ff6f697e914789","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53d6f0cf08ff6f697e914789: start","job_status":"start","queue_duration_s":0.272499,"scheduling_latency_s":0.272512} gitlab # [ 148.814456] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53d6f0cf08ff6f697e914789","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53d6f0cf08ff6f697e914789: done: 0.009674 sec","job_status":"done","queue_duration_s":0.272499,"scheduling_latency_s":0.272512,"redis_calls":13,"redis_duration_s":0.001618,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009674,"completed_at":"2026-09-23T19:19:41.321Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.815934] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c681e4b9ca80a90eb7a8a9b3","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c681e4b9ca80a90eb7a8a9b3: start","job_status":"start","queue_duration_s":0.283141,"scheduling_latency_s":0.283154} gitlab # [ 148.860789] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c681e4b9ca80a90eb7a8a9b3","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c681e4b9ca80a90eb7a8a9b3: done: 0.009615 sec","job_status":"done","queue_duration_s":0.283141,"scheduling_latency_s":0.283154,"redis_calls":13,"redis_duration_s":0.0015760000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009615,"completed_at":"2026-09-23T19:19:41.332Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.861896] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ab38f2a0c9204292855d568","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ab38f2a0c9204292855d568: start","job_status":"start","queue_duration_s":0.293571,"scheduling_latency_s":0.293584} gitlab # [ 148.871990] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ab38f2a0c9204292855d568","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ab38f2a0c9204292855d568: done: 0.009724 sec","job_status":"done","queue_duration_s":0.293571,"scheduling_latency_s":0.293584,"redis_calls":13,"redis_duration_s":0.001726,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001193,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009724,"completed_at":"2026-09-23T19:19:41.342Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.927134] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.343Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3414fe4e749bcdd2cad004a","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3414fe4e749bcdd2cad004a: start","job_status":"start","queue_duration_s":0.304153,"scheduling_latency_s":0.304166} gitlab # [ 148.928108] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3414fe4e749bcdd2cad004a","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3414fe4e749bcdd2cad004a: done: 0.009719 sec","job_status":"done","queue_duration_s":0.304153,"scheduling_latency_s":0.304166,"redis_calls":13,"redis_duration_s":0.0017369999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001223,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009719,"completed_at":"2026-09-23T19:19:41.353Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 148.974476] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.354Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7d92ff8f7a2a074c6cc3646","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7d92ff8f7a2a074c6cc3646: start","job_status":"start","queue_duration_s":0.314697,"scheduling_latency_s":0.31471} gitlab # [ 148.975434] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7d92ff8f7a2a074c6cc3646","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7d92ff8f7a2a074c6cc3646: done: 0.01001 sec","job_status":"done","queue_duration_s":0.314697,"scheduling_latency_s":0.31471,"redis_calls":13,"redis_duration_s":0.002013,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000665,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001348,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01001,"completed_at":"2026-09-23T19:19:41.364Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.014807] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bb859de2df3f33b63302014","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bb859de2df3f33b63302014: start","job_status":"start","queue_duration_s":0.32553,"scheduling_latency_s":0.325543} gitlab # [ 149.015805] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bb859de2df3f33b63302014","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bb859de2df3f33b63302014: done: 0.009855 sec","job_status":"done","queue_duration_s":0.32553,"scheduling_latency_s":0.325543,"redis_calls":13,"redis_duration_s":0.001556,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009855,"completed_at":"2026-09-23T19:19:41.374Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.064926] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.375Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7341ae082a684ee854f553b3","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7341ae082a684ee854f553b3: start","job_status":"start","queue_duration_s":0.336214,"scheduling_latency_s":0.336227} gitlab # [ 149.065946] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7341ae082a684ee854f553b3","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7341ae082a684ee854f553b3: done: 0.009584 sec","job_status":"done","queue_duration_s":0.336214,"scheduling_latency_s":0.336227,"redis_calls":13,"redis_duration_s":0.001584,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009584,"completed_at":"2026-09-23T19:19:41.385Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.093286] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cadd7b71acdcd7e008156acd","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cadd7b71acdcd7e008156acd: start","job_status":"start","queue_duration_s":0.346778,"scheduling_latency_s":0.346791} gitlab # [ 149.132950] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cadd7b71acdcd7e008156acd","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cadd7b71acdcd7e008156acd: done: 0.009503 sec","job_status":"done","queue_duration_s":0.346778,"scheduling_latency_s":0.346791,"redis_calls":13,"redis_duration_s":0.001536,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009503,"completed_at":"2026-09-23T19:19:41.395Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.139558] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.396Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5f1d5ca7695b84f602fecc1","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5f1d5ca7695b84f602fecc1: start","job_status":"start","queue_duration_s":0.357304,"scheduling_latency_s":0.357318} gitlab # [ 149.151129] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.406Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5f1d5ca7695b84f602fecc1","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5f1d5ca7695b84f602fecc1: done: 0.009569 sec","job_status":"done","queue_duration_s":0.357304,"scheduling_latency_s":0.357318,"redis_calls":13,"redis_duration_s":0.001546,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009569,"completed_at":"2026-09-23T19:19:41.406Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.219214] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0551053e70a2bc3d5039d578","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0551053e70a2bc3d5039d578: start","job_status":"start","queue_duration_s":0.41263,"scheduling_latency_s":0.412653} gitlab # [ 149.219931] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0551053e70a2bc3d5039d578","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0551053e70a2bc3d5039d578: done: 0.011331 sec","job_status":"done","queue_duration_s":0.41263,"scheduling_latency_s":0.412653,"redis_calls":13,"redis_duration_s":0.001769,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000696,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011331,"completed_at":"2026-09-23T19:19:41.463Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.232035] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b96f51943fbeda8bdff5debb","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b96f51943fbeda8bdff5debb: start","job_status":"start","queue_duration_s":0.424775,"scheduling_latency_s":0.424788} gitlab # [ 149.243163] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b96f51943fbeda8bdff5debb","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b96f51943fbeda8bdff5debb: done: 0.008764 sec","job_status":"done","queue_duration_s":0.424775,"scheduling_latency_s":0.424788,"redis_calls":13,"redis_duration_s":0.00151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008764,"completed_at":"2026-09-23T19:19:41.472Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.285761] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b3e4430bff9db986fa9fb9a","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b3e4430bff9db986fa9fb9a: start","job_status":"start","queue_duration_s":0.434374,"scheduling_latency_s":0.434387} gitlab # [ 149.288642] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.482Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b3e4430bff9db986fa9fb9a","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b3e4430bff9db986fa9fb9a: done: 0.00913 sec","job_status":"done","queue_duration_s":0.434374,"scheduling_latency_s":0.434387,"redis_calls":13,"redis_duration_s":0.001542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00913,"completed_at":"2026-09-23T19:19:41.482Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.324053] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.483Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fb5e7e6f89222695d6bb327","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fb5e7e6f89222695d6bb327: start","job_status":"start","queue_duration_s":0.444295,"scheduling_latency_s":0.444307} gitlab # [ 149.365689] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fb5e7e6f89222695d6bb327","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fb5e7e6f89222695d6bb327: done: 0.009226 sec","job_status":"done","queue_duration_s":0.444295,"scheduling_latency_s":0.444307,"redis_calls":13,"redis_duration_s":0.001518,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009226,"completed_at":"2026-09-23T19:19:41.492Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.369533] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a598d6f45ef8756e14af82b8","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a598d6f45ef8756e14af82b8: start","job_status":"start","queue_duration_s":0.45464,"scheduling_latency_s":0.454653} gitlab # [ 149.380645] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a598d6f45ef8756e14af82b8","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a598d6f45ef8756e14af82b8: done: 0.009134 sec","job_status":"done","queue_duration_s":0.45464,"scheduling_latency_s":0.454653,"redis_calls":13,"redis_duration_s":0.0015279999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009134,"completed_at":"2026-09-23T19:19:41.503Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.427575] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.504Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b26cdfb4ad112d9ec797d9e","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b26cdfb4ad112d9ec797d9e: start","job_status":"start","queue_duration_s":0.464595,"scheduling_latency_s":0.464608} gitlab # [ 149.428163] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b26cdfb4ad112d9ec797d9e","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b26cdfb4ad112d9ec797d9e: done: 0.009087 sec","job_status":"done","queue_duration_s":0.464595,"scheduling_latency_s":0.464608,"redis_calls":13,"redis_duration_s":0.001814,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001295,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009087,"completed_at":"2026-09-23T19:19:41.513Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.475191] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.514Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d5ac273934609fd9f457d14","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d5ac273934609fd9f457d14: start","job_status":"start","queue_duration_s":0.474572,"scheduling_latency_s":0.474584} gitlab # [ 149.475919] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d5ac273934609fd9f457d14","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d5ac273934609fd9f457d14: done: 0.009036 sec","job_status":"done","queue_duration_s":0.474572,"scheduling_latency_s":0.474584,"redis_calls":13,"redis_duration_s":0.001517,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009036,"completed_at":"2026-09-23T19:19:41.523Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.506655] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"353420a3fd1b92affd694695","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-353420a3fd1b92affd694695: start","job_status":"start","queue_duration_s":0.484394,"scheduling_latency_s":0.484406} gitlab # [ 149.517717] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.533Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"353420a3fd1b92affd694695","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-353420a3fd1b92affd694695: done: 0.009253 sec","job_status":"done","queue_duration_s":0.484394,"scheduling_latency_s":0.484406,"redis_calls":13,"redis_duration_s":0.001503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000484,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009253,"completed_at":"2026-09-23T19:19:41.533Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.557571] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19f85c55960df87de45095c7","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19f85c55960df87de45095c7: start","job_status":"start","queue_duration_s":0.49486,"scheduling_latency_s":0.494874} gitlab # [ 149.562836] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.543Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19f85c55960df87de45095c7","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19f85c55960df87de45095c7: done: 0.009171 sec","job_status":"done","queue_duration_s":0.49486,"scheduling_latency_s":0.494874,"redis_calls":13,"redis_duration_s":0.0015370000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001043,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009171,"completed_at":"2026-09-23T19:19:41.543Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.610369] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.544Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"776b75dd34839754296f88e7","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-776b75dd34839754296f88e7: start","job_status":"start","queue_duration_s":0.50485,"scheduling_latency_s":0.504863} gitlab # [ 149.611215] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.553Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"776b75dd34839754296f88e7","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-776b75dd34839754296f88e7: done: 0.009209 sec","job_status":"done","queue_duration_s":0.50485,"scheduling_latency_s":0.504863,"redis_calls":13,"redis_duration_s":0.0015769999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009209,"completed_at":"2026-09-23T19:19:41.553Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.676781] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac2e6de2998cd992ba78dd67","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac2e6de2998cd992ba78dd67: start","job_status":"start","queue_duration_s":0.514865,"scheduling_latency_s":0.514878} gitlab # [ 149.677889] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac2e6de2998cd992ba78dd67","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac2e6de2998cd992ba78dd67: done: 0.010409 sec","job_status":"done","queue_duration_s":0.514865,"scheduling_latency_s":0.514878,"redis_calls":13,"redis_duration_s":0.001795,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001297,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010409,"completed_at":"2026-09-23T19:19:41.564Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.726417] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc2d5ffee710051dd4a45dda","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc2d5ffee710051dd4a45dda: start","job_status":"start","queue_duration_s":0.526112,"scheduling_latency_s":0.526125} gitlab # [ 149.727325] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.576Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc2d5ffee710051dd4a45dda","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc2d5ffee710051dd4a45dda: done: 0.01128 sec","job_status":"done","queue_duration_s":0.526112,"scheduling_latency_s":0.526125,"redis_calls":13,"redis_duration_s":0.0016549999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001117,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01128,"completed_at":"2026-09-23T19:19:41.576Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.736924] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f117ae05fa57e9e9ecf831be","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f117ae05fa57e9e9ecf831be: start","job_status":"start","queue_duration_s":0.538234,"scheduling_latency_s":0.538246} gitlab # [ 149.748807] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.587Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f117ae05fa57e9e9ecf831be","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f117ae05fa57e9e9ecf831be: done: 0.009329 sec","job_status":"done","queue_duration_s":0.538234,"scheduling_latency_s":0.538246,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000498,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009329,"completed_at":"2026-09-23T19:19:41.587Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.801435] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.587Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f64437220f8f76615930a630","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f64437220f8f76615930a630: start","job_status":"start","queue_duration_s":0.548352,"scheduling_latency_s":0.548365} gitlab # [ 149.802300] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f64437220f8f76615930a630","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f64437220f8f76615930a630: done: 0.009181 sec","job_status":"done","queue_duration_s":0.548352,"scheduling_latency_s":0.548365,"redis_calls":13,"redis_duration_s":0.0015019999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009181,"completed_at":"2026-09-23T19:19:41.596Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.852682] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcd1189c17e3da07a8fa6102","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcd1189c17e3da07a8fa6102: start","job_status":"start","queue_duration_s":0.558771,"scheduling_latency_s":0.558784} gitlab # [ 149.853738] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.607Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcd1189c17e3da07a8fa6102","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcd1189c17e3da07a8fa6102: done: 0.009173 sec","job_status":"done","queue_duration_s":0.558771,"scheduling_latency_s":0.558784,"redis_calls":13,"redis_duration_s":0.00154,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009173,"completed_at":"2026-09-23T19:19:41.607Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.905463] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"646571dfca38bf1687f9c4de","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-646571dfca38bf1687f9c4de: start","job_status":"start","queue_duration_s":0.56876,"scheduling_latency_s":0.568773} gitlab # [ 149.906329] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"646571dfca38bf1687f9c4de","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-646571dfca38bf1687f9c4de: done: 0.009165 sec","job_status":"done","queue_duration_s":0.56876,"scheduling_latency_s":0.568773,"redis_calls":13,"redis_duration_s":0.001541,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001005,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009165,"completed_at":"2026-09-23T19:19:41.617Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.950890] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1609cf1099fcf820b4447f6a","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1609cf1099fcf820b4447f6a: start","job_status":"start","queue_duration_s":0.578751,"scheduling_latency_s":0.578765} gitlab # [ 149.951928] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.627Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1609cf1099fcf820b4447f6a","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1609cf1099fcf820b4447f6a: done: 0.009194 sec","job_status":"done","queue_duration_s":0.578751,"scheduling_latency_s":0.578765,"redis_calls":13,"redis_duration_s":0.001499,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009194,"completed_at":"2026-09-23T19:19:41.627Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 149.969114] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.628Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c9f07bf5614a889e3bbf2ba","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c9f07bf5614a889e3bbf2ba: start","job_status":"start","queue_duration_s":0.589233,"scheduling_latency_s":0.589245} gitlab # [ 150.012705] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.638Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c9f07bf5614a889e3bbf2ba","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c9f07bf5614a889e3bbf2ba: done: 0.009513 sec","job_status":"done","queue_duration_s":0.589233,"scheduling_latency_s":0.589245,"redis_calls":13,"redis_duration_s":0.001493,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009513,"completed_at":"2026-09-23T19:19:41.638Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.015082] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d555035bbb2c0560f242ac5","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d555035bbb2c0560f242ac5: start","job_status":"start","queue_duration_s":0.59967,"scheduling_latency_s":0.599682} gitlab # [ 150.063228] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d555035bbb2c0560f242ac5","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d555035bbb2c0560f242ac5: done: 0.00934 sec","job_status":"done","queue_duration_s":0.59967,"scheduling_latency_s":0.599682,"redis_calls":13,"redis_duration_s":0.001549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000497,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00934,"completed_at":"2026-09-23T19:19:41.648Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.064721] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d207b066974b7529cb6f1ea","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d207b066974b7529cb6f1ea: start","job_status":"start","queue_duration_s":0.609885,"scheduling_latency_s":0.609899} gitlab # [ 150.106438] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d207b066974b7529cb6f1ea","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d207b066974b7529cb6f1ea: done: 0.009374 sec","job_status":"done","queue_duration_s":0.609885,"scheduling_latency_s":0.609899,"redis_calls":13,"redis_duration_s":0.001518,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009374,"completed_at":"2026-09-23T19:19:41.658Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.107918] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5c910f6665c292545a75a08","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5c910f6665c292545a75a08: start","job_status":"start","queue_duration_s":0.620077,"scheduling_latency_s":0.62009} gitlab # [ 150.118696] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5c910f6665c292545a75a08","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5c910f6665c292545a75a08: done: 0.00981 sec","job_status":"done","queue_duration_s":0.620077,"scheduling_latency_s":0.62009,"redis_calls":13,"redis_duration_s":0.001567,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00981,"completed_at":"2026-09-23T19:19:41.669Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.164303] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.670Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e00e923a2c1b3a256265e274","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e00e923a2c1b3a256265e274: start","job_status":"start","queue_duration_s":0.6313,"scheduling_latency_s":0.631312} gitlab # [ 150.165187] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e00e923a2c1b3a256265e274","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e00e923a2c1b3a256265e274: done: 0.009348 sec","job_status":"done","queue_duration_s":0.6313,"scheduling_latency_s":0.631312,"redis_calls":13,"redis_duration_s":0.001505,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009348,"completed_at":"2026-09-23T19:19:41.680Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.213756] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04a52f5cfaf681c67a148c6c","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04a52f5cfaf681c67a148c6c: start","job_status":"start","queue_duration_s":0.641556,"scheduling_latency_s":0.641569} gitlab # [ 150.214835] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.690Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04a52f5cfaf681c67a148c6c","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04a52f5cfaf681c67a148c6c: done: 0.00929 sec","job_status":"done","queue_duration_s":0.641556,"scheduling_latency_s":0.641569,"redis_calls":13,"redis_duration_s":0.0015270000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00929,"completed_at":"2026-09-23T19:19:41.690Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.266831] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b521e99b61b6dc0f7329f1d","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b521e99b61b6dc0f7329f1d: start","job_status":"start","queue_duration_s":0.651669,"scheduling_latency_s":0.651681} gitlab # [ 150.267910] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b521e99b61b6dc0f7329f1d","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b521e99b61b6dc0f7329f1d: done: 0.009224 sec","job_status":"done","queue_duration_s":0.651669,"scheduling_latency_s":0.651681,"redis_calls":13,"redis_duration_s":0.001512,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009224,"completed_at":"2026-09-23T19:19:41.700Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.311862] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.701Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fcc7b8d3816e13fe486ab46","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fcc7b8d3816e13fe486ab46: start","job_status":"start","queue_duration_s":0.661725,"scheduling_latency_s":0.661739} gitlab # [ 150.312893] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fcc7b8d3816e13fe486ab46","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fcc7b8d3816e13fe486ab46: done: 0.009167 sec","job_status":"done","queue_duration_s":0.661725,"scheduling_latency_s":0.661739,"redis_calls":13,"redis_duration_s":0.001503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009167,"completed_at":"2026-09-23T19:19:41.710Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.361388] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdb57347716e686fafc53ce9","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdb57347716e686fafc53ce9: start","job_status":"start","queue_duration_s":0.67208,"scheduling_latency_s":0.672092} gitlab # [ 150.362276] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fdb57347716e686fafc53ce9","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fdb57347716e686fafc53ce9: done: 0.009211 sec","job_status":"done","queue_duration_s":0.67208,"scheduling_latency_s":0.672092,"redis_calls":13,"redis_duration_s":0.001562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009211,"completed_at":"2026-09-23T19:19:41.720Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.414301] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e130d087dd97db343bac697c","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e130d087dd97db343bac697c: start","job_status":"start","queue_duration_s":0.682106,"scheduling_latency_s":0.682118} gitlab # [ 150.415285] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e130d087dd97db343bac697c","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e130d087dd97db343bac697c: done: 0.009257 sec","job_status":"done","queue_duration_s":0.682106,"scheduling_latency_s":0.682118,"redis_calls":13,"redis_duration_s":0.0015270000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009257,"completed_at":"2026-09-23T19:19:41.730Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.429830] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a35593645ca4cc347a72e1be","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a35593645ca4cc347a72e1be: start","job_status":"start","queue_duration_s":0.692246,"scheduling_latency_s":0.69226} gitlab # [ 150.473102] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a35593645ca4cc347a72e1be","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a35593645ca4cc347a72e1be: done: 0.009165 sec","job_status":"done","queue_duration_s":0.692246,"scheduling_latency_s":0.69226,"redis_calls":13,"redis_duration_s":0.001479,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009165,"completed_at":"2026-09-23T19:19:41.740Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.475779] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7c1354fc968446a2c3d02a7","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7c1354fc968446a2c3d02a7: start","job_status":"start","queue_duration_s":0.702237,"scheduling_latency_s":0.70225} gitlab # [ 150.487789] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7c1354fc968446a2c3d02a7","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7c1354fc968446a2c3d02a7: done: 0.009158 sec","job_status":"done","queue_duration_s":0.702237,"scheduling_latency_s":0.70225,"redis_calls":13,"redis_duration_s":0.001487,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009158,"completed_at":"2026-09-23T19:19:41.750Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.535070] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.752Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a47eea09249ca7da02c9fa93","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a47eea09249ca7da02c9fa93: start","job_status":"start","queue_duration_s":0.712591,"scheduling_latency_s":0.712603} gitlab # [ 150.535997] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.761Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a47eea09249ca7da02c9fa93","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a47eea09249ca7da02c9fa93: done: 0.009244 sec","job_status":"done","queue_duration_s":0.712591,"scheduling_latency_s":0.712603,"redis_calls":13,"redis_duration_s":0.001555,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009244,"completed_at":"2026-09-23T19:19:41.761Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.598249] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.762Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"888f208acc0140cff4f70ad3","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-888f208acc0140cff4f70ad3: start","job_status":"start","queue_duration_s":0.72281,"scheduling_latency_s":0.722823} gitlab # [ 150.598946] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.771Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"888f208acc0140cff4f70ad3","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-888f208acc0140cff4f70ad3: done: 0.009258 sec","job_status":"done","queue_duration_s":0.72281,"scheduling_latency_s":0.722823,"redis_calls":13,"redis_duration_s":0.001784,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001297,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009258,"completed_at":"2026-09-23T19:19:41.771Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.643937] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec2fde654871665c9c445a8b","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec2fde654871665c9c445a8b: start","job_status":"start","queue_duration_s":0.732878,"scheduling_latency_s":0.73289} gitlab # [ 150.644959] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.781Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ec2fde654871665c9c445a8b","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ec2fde654871665c9c445a8b: done: 0.009239 sec","job_status":"done","queue_duration_s":0.732878,"scheduling_latency_s":0.73289,"redis_calls":13,"redis_duration_s":0.0015040000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009239,"completed_at":"2026-09-23T19:19:41.781Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.685898] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.782Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f69125005047ad1586c2e013","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f69125005047ad1586c2e013: start","job_status":"start","queue_duration_s":0.74294,"scheduling_latency_s":0.742953} gitlab # [ 150.686803] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f69125005047ad1586c2e013","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f69125005047ad1586c2e013: done: 0.00898 sec","job_status":"done","queue_duration_s":0.74294,"scheduling_latency_s":0.742953,"redis_calls":13,"redis_duration_s":0.001549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005553,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00898,"completed_at":"2026-09-23T19:19:41.791Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.707728] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.792Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8e2f0f28b06d24acfef0839","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8e2f0f28b06d24acfef0839: start","job_status":"start","queue_duration_s":0.753148,"scheduling_latency_s":0.753161} gitlab # [ 150.752505] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.801Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8e2f0f28b06d24acfef0839","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8e2f0f28b06d24acfef0839: done: 0.00914 sec","job_status":"done","queue_duration_s":0.753148,"scheduling_latency_s":0.753161,"redis_calls":13,"redis_duration_s":0.001492,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00914,"completed_at":"2026-09-23T19:19:41.801Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.754077] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62a270600db76497bc86e8c2","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62a270600db76497bc86e8c2: start","job_status":"start","queue_duration_s":0.763105,"scheduling_latency_s":0.763117} gitlab # [ 150.765158] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62a270600db76497bc86e8c2","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62a270600db76497bc86e8c2: done: 0.009218 sec","job_status":"done","queue_duration_s":0.763105,"scheduling_latency_s":0.763117,"redis_calls":13,"redis_duration_s":0.001592,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009218,"completed_at":"2026-09-23T19:19:41.811Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 150.806360] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7a0251d210789fa4976a770","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7a0251d210789fa4976a770: start","job_status":"start","queue_duration_s":0.773199,"scheduling_latency_s":0.773212} gitlab # [ 150.810869] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:19:41.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7a0251d210789fa4976a770","created_at":"2026-09-23T19:19:40.994Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-162d78d1490d4245","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"89524c30563fcea68b71b74db7b905f4","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:19:41.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7a0251d210789fa4976a770: done: 0.013782 sec","job_status":"done","queue_duration_s":0.773199,"scheduling_latency_s":0.773212,"redis_calls":13,"redis_duration_s":0.004012,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003495,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013782,"completed_at":"2026-09-23T19:19:41.826Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 151.035591] clocksource: Long readout interval, skipping watchdog check: cs_nsec: 2610988351 wd_nsec: 2610988354 gitlab # [ 160.960282] setup-gitlab-start[2218]: W, [2026-09-23T19:19:54.232150 #2218] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 171.400640] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:04.672Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"7d3cd321d20bb21b1c02b79c","created_at":"2026-09-23T19:20:04.621Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b9dd41e48ffa2c7d86398c9458e0c1c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.669Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-7d3cd321d20bb21b1c02b79c: start","job_status":"start","queue_duration_s":0.002974,"scheduling_latency_s":0.003024} gitlab # [ 172.203867] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"7d3cd321d20bb21b1c02b79c","created_at":"2026-09-23T19:20:04.621Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b9dd41e48ffa2c7d86398c9458e0c1c","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.669Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-7d3cd321d20bb21b1c02b79c: done: 0.804363 sec","job_status":"done","queue_duration_s":0.002974,"scheduling_latency_s":0.003024,"redis_calls":562,"redis_duration_s":0.462521,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001576,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.460945,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.378018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.804363,"completed_at":"2026-09-23T19:20:05.476Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.276990] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"f0ae723c3babd865a8535be9","created_at":"2026-09-23T19:20:04.705Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9fef116aaf5f6458b2b52d2f29ab61f0","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.717Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-f0ae723c3babd865a8535be9: start","job_status":"start","queue_duration_s":0.76031,"scheduling_latency_s":0.760331} gitlab # [ 172.289785] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.492Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"f0ae723c3babd865a8535be9","created_at":"2026-09-23T19:20:04.705Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9fef116aaf5f6458b2b52d2f29ab61f0","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.717Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-f0ae723c3babd865a8535be9: done: 0.014778 sec","job_status":"done","queue_duration_s":0.76031,"scheduling_latency_s":0.760331,"redis_calls":11,"redis_duration_s":0.002482,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000442,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001269,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00034,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000431,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008123,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000277968,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.014778,"completed_at":"2026-09-23T19:20:05.492Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000741,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.328511] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.493Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"c2bf3740a10cfe41167cbb6a","created_at":"2026-09-23T19:20:04.736Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64cf2349ed661ea8eedd087bc58d0a53","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.740Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-c2bf3740a10cfe41167cbb6a: start","job_status":"start","queue_duration_s":0.75282,"scheduling_latency_s":0.752833} gitlab # [ 172.341591] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.499Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 172.344759] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.502Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"c2bf3740a10cfe41167cbb6a","created_at":"2026-09-23T19:20:04.736Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64cf2349ed661ea8eedd087bc58d0a53","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.740Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-c2bf3740a10cfe41167cbb6a: done: 0.008825 sec","job_status":"done","queue_duration_s":0.75282,"scheduling_latency_s":0.752833,"redis_calls":7,"redis_duration_s":0.001519,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001087,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000432,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008825,"completed_at":"2026-09-23T19:20:05.502Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.345617] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.504Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"03033ab6d05602822c0e2373","created_at":"2026-09-23T19:20:04.750Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"833faa455c38103f7a85b661718acaab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.756Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-03033ab6d05602822c0e2373: start","job_status":"start","queue_duration_s":0.748526,"scheduling_latency_s":0.748579} gitlab # [ 172.353085] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.534Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"03033ab6d05602822c0e2373","created_at":"2026-09-23T19:20:04.750Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"833faa455c38103f7a85b661718acaab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.756Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-03033ab6d05602822c0e2373: done: 0.030278 sec","job_status":"done","queue_duration_s":0.748526,"scheduling_latency_s":0.748579,"redis_calls":14,"redis_duration_s":0.003181,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000291,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002257,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000633,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030278,"completed_at":"2026-09-23T19:20:05.534Z","load_balancing_strategy":"primary","db_duration_s":0.000719,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.411660] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.535Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"821ef3e61d604b4b38d3275e","created_at":"2026-09-23T19:20:04.767Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0b5eb6d89e8362b86db0ae1921a3f28","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.772Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-821ef3e61d604b4b38d3275e: start","job_status":"start","queue_duration_s":0.763523,"scheduling_latency_s":0.763539} gitlab # [ 172.412421] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.546Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"821ef3e61d604b4b38d3275e","created_at":"2026-09-23T19:20:04.767Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0b5eb6d89e8362b86db0ae1921a3f28","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.772Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-821ef3e61d604b4b38d3275e: done: 0.010274 sec","job_status":"done","queue_duration_s":0.763523,"scheduling_latency_s":0.763539,"redis_calls":7,"redis_duration_s":0.001498,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001111,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000387,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010274,"completed_at":"2026-09-23T19:20:05.546Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00063,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.452592] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.547Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"d6cd9a9aca14499091112789","created_at":"2026-09-23T19:20:04.784Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d5de68e4fa0354207d157fd492294c7","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.794Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-d6cd9a9aca14499091112789: start","job_status":"start","queue_duration_s":0.752696,"scheduling_latency_s":0.75271} gitlab # [ 172.453484] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.582Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"d6cd9a9aca14499091112789","created_at":"2026-09-23T19:20:04.784Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d5de68e4fa0354207d157fd492294c7","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.794Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-d6cd9a9aca14499091112789: done: 0.034484 sec","job_status":"done","queue_duration_s":0.752696,"scheduling_latency_s":0.75271,"redis_calls":10,"redis_duration_s":0.002523,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000539,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001481,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000503,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000471568,"duration_s":0.034484,"completed_at":"2026-09-23T19:20:05.582Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002612,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.506723] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.583Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"0078d8a88ffbc5bca63cfbdf","created_at":"2026-09-23T19:20:04.805Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6254b8c46b99596edb4c374c775bfc85","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.816Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-0078d8a88ffbc5bca63cfbdf: start","job_status":"start","queue_duration_s":0.767479,"scheduling_latency_s":0.767504} gitlab # [ 172.507487] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.604Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"0078d8a88ffbc5bca63cfbdf","created_at":"2026-09-23T19:20:04.805Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6254b8c46b99596edb4c374c775bfc85","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.816Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-0078d8a88ffbc5bca63cfbdf: done: 0.021109 sec","job_status":"done","queue_duration_s":0.767479,"scheduling_latency_s":0.767504,"redis_calls":10,"redis_duration_s":0.001555,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000349,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000907,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000217905,"duration_s":0.021109,"completed_at":"2026-09-23T19:20:05.604Z","load_balancing_strategy":"primary","db_duration_s":0.000356,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.552768] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.605Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"8745ced123ca343946b4c713","created_at":"2026-09-23T19:20:04.826Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd0934981a2e93c3ca86c212d2d6c1d2","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.830Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-8745ced123ca343946b4c713: start","job_status":"start","queue_duration_s":0.775084,"scheduling_latency_s":0.775098} gitlab # [ 172.553610] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.615Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"8745ced123ca343946b4c713","created_at":"2026-09-23T19:20:04.826Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd0934981a2e93c3ca86c212d2d6c1d2","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.830Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-8745ced123ca343946b4c713: done: 0.009515 sec","job_status":"done","queue_duration_s":0.775084,"scheduling_latency_s":0.775098,"redis_calls":9,"redis_duration_s":0.001144,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000625,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000519,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000280482,"exclusive_lock_hold_duration_s":0.001237028,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.009515,"completed_at":"2026-09-23T19:20:05.615Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000317,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.598210] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.615Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"40d1c52a45155f7e6f6553d0","created_at":"2026-09-23T19:20:04.865Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2b37262583e1d5c35ca2fe111439773","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.870Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-40d1c52a45155f7e6f6553d0: start","job_status":"start","queue_duration_s":0.745424,"scheduling_latency_s":0.745436} gitlab # [ 172.598868] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.708Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 172.601063] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.712Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"40d1c52a45155f7e6f6553d0","created_at":"2026-09-23T19:20:04.865Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2b37262583e1d5c35ca2fe111439773","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.870Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-40d1c52a45155f7e6f6553d0: done: 0.096328 sec","job_status":"done","queue_duration_s":0.745424,"scheduling_latency_s":0.745436,"redis_calls":7,"redis_duration_s":0.08542,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000799,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.084621,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.096328,"completed_at":"2026-09-23T19:20:05.712Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.637752] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.713Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"a0bee86357b9a9f54fcb87bf","created_at":"2026-09-23T19:20:04.899Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5dcc371370f5957eb8f66f4e5c406b4e","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.910Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-a0bee86357b9a9f54fcb87bf: start","job_status":"start","queue_duration_s":0.802889,"scheduling_latency_s":0.802909} gitlab # [ 172.649770] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.755Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"a0bee86357b9a9f54fcb87bf","created_at":"2026-09-23T19:20:04.899Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5dcc371370f5957eb8f66f4e5c406b4e","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.910Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-a0bee86357b9a9f54fcb87bf: done: 0.04235 sec","job_status":"done","queue_duration_s":0.802889,"scheduling_latency_s":0.802909,"redis_calls":10,"redis_duration_s":0.001533,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000265,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000313,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000335239,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.003181969},"duration_s":0.04235,"completed_at":"2026-09-23T19:20:05.755Z","load_balancing_strategy":"primary","db_duration_s":0.014547,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.699114] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.756Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7b769c7681aea723a9ce981f","created_at":"2026-09-23T19:20:04.922Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b193d10239d9be2fb5fa63901897f996","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.926Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7b769c7681aea723a9ce981f: start","job_status":"start","queue_duration_s":0.829988,"scheduling_latency_s":0.830002} gitlab # [ 172.700574] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.764Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7b769c7681aea723a9ce981f","created_at":"2026-09-23T19:20:04.922Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b193d10239d9be2fb5fa63901897f996","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.926Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7b769c7681aea723a9ce981f: done: 0.008109 sec","job_status":"done","queue_duration_s":0.829988,"scheduling_latency_s":0.830002,"redis_calls":7,"redis_duration_s":0.000831,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000603,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000228,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008109,"completed_at":"2026-09-23T19:20:05.764Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.764616] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.765Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"f42f4821ca38c35bc41fee23","created_at":"2026-09-23T19:20:04.942Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"697aedafab6b8ad5bd3b3c816b1cb553","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-23T19:20:04.946Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-f42f4821ca38c35bc41fee23: start","job_status":"start","queue_duration_s":0.819346,"scheduling_latency_s":0.819359} gitlab # [ 172.765428] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.770Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 172.766150] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.773Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"f42f4821ca38c35bc41fee23","created_at":"2026-09-23T19:20:04.942Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"697aedafab6b8ad5bd3b3c816b1cb553","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-23T19:20:04.946Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-f42f4821ca38c35bc41fee23: done: 0.007447 sec","job_status":"done","queue_duration_s":0.819346,"scheduling_latency_s":0.819359,"redis_calls":7,"redis_duration_s":0.00081,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000231,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004814,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007447,"completed_at":"2026-09-23T19:20:05.773Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.815371] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.774Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"b0849a5a5870679754dd7811","created_at":"2026-09-23T19:20:04.956Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4654ada86fb9baf5490f8e45306099d8","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.965Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-b0849a5a5870679754dd7811: start","job_status":"start","queue_duration_s":0.808112,"scheduling_latency_s":0.808126} gitlab # [ 172.816105] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.788Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"b0849a5a5870679754dd7811","created_at":"2026-09-23T19:20:04.956Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4654ada86fb9baf5490f8e45306099d8","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:04.965Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-b0849a5a5870679754dd7811: done: 0.014898 sec","job_status":"done","queue_duration_s":0.808112,"scheduling_latency_s":0.808126,"redis_calls":12,"redis_duration_s":0.001768,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000473,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000309,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000729,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000257,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000222654,"duration_s":0.014898,"completed_at":"2026-09-23T19:20:05.788Z","load_balancing_strategy":"primary","db_duration_s":0.003034,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.856785] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.789Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"46a9d973510e94ecda5a345f","created_at":"2026-09-23T19:20:04.979Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1a861f0de6cd25827da9416ac512707","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-23T19:20:04.984Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-46a9d973510e94ecda5a345f: start","job_status":"start","queue_duration_s":0.805452,"scheduling_latency_s":0.805465} gitlab # [ 172.857718] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.801Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"46a9d973510e94ecda5a345f","created_at":"2026-09-23T19:20:04.979Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1a861f0de6cd25827da9416ac512707","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-23T19:20:04.984Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-46a9d973510e94ecda5a345f: done: 0.012166 sec","job_status":"done","queue_duration_s":0.805452,"scheduling_latency_s":0.805465,"redis_calls":7,"redis_duration_s":0.000894,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012166,"completed_at":"2026-09-23T19:20:05.801Z","load_balancing_strategy":"primary","db_duration_s":0.000498,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.903499] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.802Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"RepositoryCheck::DispatchWorker","jid":"714777e995d63511408e5994","created_at":"2026-09-23T19:20:04.996Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89d2d5558af3c5678c4e711eabcca62a","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:bce0277309dfde25386eb0f6b49415f8ec269d6e1b0147ad102f2a38c86bd6e2","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.005Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryCheck::DispatchWorker JID-714777e995d63511408e5994: start","job_status":"start","queue_duration_s":0.797149,"scheduling_latency_s":0.797162} gitlab # [ 172.904428] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.833Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"RepositoryCheck::DispatchWorker","jid":"714777e995d63511408e5994","created_at":"2026-09-23T19:20:04.996Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"89d2d5558af3c5678c4e711eabcca62a","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:bce0277309dfde25386eb0f6b49415f8ec269d6e1b0147ad102f2a38c86bd6e2","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.005Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryCheck::DispatchWorker JID-714777e995d63511408e5994: done: 0.030501 sec","job_status":"done","queue_duration_s":0.797149,"scheduling_latency_s":0.797162,"gitaly_calls":1,"gitaly_duration_s":0.009717,"redis_calls":26,"redis_duration_s":0.003455,"redis_read_bytes":45,"redis_write_bytes":3469,"redis_cache_calls":5,"redis_cache_duration_s":0.000692,"redis_cache_read_bytes":23,"redis_cache_write_bytes":95,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001144,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1190,"redis_queues_calls":2,"redis_queues_duration_s":0.000178,"redis_queues_read_bytes":3,"redis_queues_write_bytes":917,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":758,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000553,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":509,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017626,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.016416891,"duration_s":0.030501,"completed_at":"2026-09-23T19:20:05.833Z","load_balancing_strategy":"primary","db_duration_s":0.001209,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 172.947832] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.834Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"76beaf09487780c4f0322c5e","created_at":"2026-09-23T19:20:05.014Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"415b69a9c808596ba90b5765b8fc0044","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.021Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-76beaf09487780c4f0322c5e: start","job_status":"start","queue_duration_s":0.812995,"scheduling_latency_s":0.813009} gitlab # [ 172.948807] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.846Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"76beaf09487780c4f0322c5e","created_at":"2026-09-23T19:20:05.014Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"415b69a9c808596ba90b5765b8fc0044","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.021Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-76beaf09487780c4f0322c5e: done: 0.012144 sec","job_status":"done","queue_duration_s":0.812995,"scheduling_latency_s":0.813009,"redis_calls":10,"redis_duration_s":0.001297,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000517,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00239388,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.012144,"completed_at":"2026-09-23T19:20:05.846Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000834,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 173.013198] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.847Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::CleanupPackageRegistryWorker","jid":"240b8736afb69ba15a5e66bb","created_at":"2026-09-23T19:20:05.030Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13fe0b4857ab3e20be25ebd15e08ef5b","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:31163b2078644913f89feb2245c5fa62ac9118d29e04cf14ce0b5def57bba8ea","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::CleanupPackageRegistryWorker JID-240b8736afb69ba15a5e66bb: start","job_status":"start","queue_duration_s":0.807847,"scheduling_latency_s":0.80786} gitlab # [ 173.013882] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.951Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::CleanupPackageRegistryWorker","jid":"240b8736afb69ba15a5e66bb","created_at":"2026-09-23T19:20:05.030Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13fe0b4857ab3e20be25ebd15e08ef5b","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:31163b2078644913f89feb2245c5fa62ac9118d29e04cf14ce0b5def57bba8ea","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.039Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::CleanupPackageRegistryWorker JID-240b8736afb69ba15a5e66bb: done: 0.103963 sec","job_status":"done","queue_duration_s":0.807847,"scheduling_latency_s":0.80786,"redis_calls":10,"redis_duration_s":0.0027229999999999997,"redis_read_bytes":9,"redis_write_bytes":1125,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000245,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001483,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000995,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":10,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.071,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.071,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025087,"extra.packages_cleanup_package_registry_worker.pending_destruction_package_files_count":0,"extra.packages_cleanup_package_registry_worker.processing_package_files_count":0,"extra.packages_cleanup_package_registry_worker.error_package_files_count":0,"extra.packages_cleanup_package_registry_worker.pending_cleanup_policies_count":0,"duration_s":0.103963,"completed_at":"2026-09-23T19:20:05.951Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.076109,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.056323] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.954Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d0e15f74500b2269c3b65000","created_at":"2026-09-23T19:20:05.048Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a891b40fb929cb3aa0fd1d922e2349ff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.052Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d0e15f74500b2269c3b65000: start","job_status":"start","queue_duration_s":0.902423,"scheduling_latency_s":0.902438} gitlab # [ 173.056737] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.959Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 173.057248] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.963Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d0e15f74500b2269c3b65000","created_at":"2026-09-23T19:20:05.048Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a891b40fb929cb3aa0fd1d922e2349ff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.052Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d0e15f74500b2269c3b65000: done: 0.008887 sec","job_status":"done","queue_duration_s":0.902423,"scheduling_latency_s":0.902438,"redis_calls":7,"redis_duration_s":0.000934,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000679,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008887,"completed_at":"2026-09-23T19:20:05.963Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.088638] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.964Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"6d26ff5256ba584f7b311130","created_at":"2026-09-23T19:20:05.061Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16a66137677989ba56ddf6383de98f79","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.064Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-6d26ff5256ba584f7b311130: start","job_status":"start","queue_duration_s":0.899566,"scheduling_latency_s":0.899579} gitlab # [ 173.101582] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.969Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 173.104589] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.972Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"6d26ff5256ba584f7b311130","created_at":"2026-09-23T19:20:05.061Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16a66137677989ba56ddf6383de98f79","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.064Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-6d26ff5256ba584f7b311130: done: 0.007849 sec","job_status":"done","queue_duration_s":0.899566,"scheduling_latency_s":0.899579,"redis_calls":7,"redis_duration_s":0.000891,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000241,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007849,"completed_at":"2026-09-23T19:20:05.972Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.166653] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.973Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ba2f85756a7f47fbef11bdd9","created_at":"2026-09-23T19:20:05.080Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f7f5b14d99f533acd058ea22d4a06c3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.089Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ba2f85756a7f47fbef11bdd9: start","job_status":"start","queue_duration_s":0.883237,"scheduling_latency_s":0.88325} gitlab # [ 173.167408] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.987Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ba2f85756a7f47fbef11bdd9","created_at":"2026-09-23T19:20:05.080Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f7f5b14d99f533acd058ea22d4a06c3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.089Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ba2f85756a7f47fbef11bdd9: done: 0.014299 sec","job_status":"done","queue_duration_s":0.883237,"scheduling_latency_s":0.88325,"redis_calls":12,"redis_duration_s":0.001589,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000247,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000571,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000296965,"exclusive_lock_hold_duration_s":0.0028836070000000003,"duration_s":0.014299,"completed_at":"2026-09-23T19:20:05.987Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001593,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.219467] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:05.988Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"ca0cf36cccdd1b2b1cb092be","created_at":"2026-09-23T19:20:05.098Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b129c5779a276627c5e201c7e3e1d1a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.102Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-ca0cf36cccdd1b2b1cb092be: start","job_status":"start","queue_duration_s":0.885913,"scheduling_latency_s":0.885926} gitlab # [ 173.220101] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.006Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"ca0cf36cccdd1b2b1cb092be","created_at":"2026-09-23T19:20:05.098Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b129c5779a276627c5e201c7e3e1d1a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.102Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-ca0cf36cccdd1b2b1cb092be: done: 0.018257 sec","job_status":"done","queue_duration_s":0.885913,"scheduling_latency_s":0.885926,"redis_calls":9,"redis_duration_s":0.001214,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00067,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000544,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010707,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000324901,"exclusive_lock_hold_duration_s":0.009896229,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.018257,"completed_at":"2026-09-23T19:20:06.006Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004887,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.268874] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.007Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"599d651104387a51453fdda2","created_at":"2026-09-23T19:20:05.112Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f78cf4685b49e6b6837ea6eb96413986","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.121Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-599d651104387a51453fdda2: start","job_status":"start","queue_duration_s":0.886382,"scheduling_latency_s":0.886396} gitlab # [ 173.269636] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.019Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"599d651104387a51453fdda2","created_at":"2026-09-23T19:20:05.112Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f78cf4685b49e6b6837ea6eb96413986","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.121Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-599d651104387a51453fdda2: done: 0.012156 sec","job_status":"done","queue_duration_s":0.886382,"scheduling_latency_s":0.886396,"redis_calls":10,"redis_duration_s":0.0012540000000000001,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000266,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000729,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000259,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000195835,"duration_s":0.012156,"completed_at":"2026-09-23T19:20:06.019Z","load_balancing_strategy":"primary","db_duration_s":0.000812,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.310853] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.020Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"f7d6c3f49c8f07056e9aee62","created_at":"2026-09-23T19:20:05.132Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8b327d4c4b29fddcf3969603d8e455e","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.141Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-f7d6c3f49c8f07056e9aee62: start","job_status":"start","queue_duration_s":0.879083,"scheduling_latency_s":0.879096} gitlab # [ 173.311659] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.035Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"f7d6c3f49c8f07056e9aee62","created_at":"2026-09-23T19:20:05.132Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8b327d4c4b29fddcf3969603d8e455e","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.141Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-f7d6c3f49c8f07056e9aee62: done: 0.014913 sec","job_status":"done","queue_duration_s":0.879083,"scheduling_latency_s":0.879096,"redis_calls":13,"redis_duration_s":0.001616,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000257,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000736,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000623,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000195276,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.014913,"completed_at":"2026-09-23T19:20:06.035Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002338,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.368680] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.036Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"827ba728d70d0e4d608d9223","created_at":"2026-09-23T19:20:05.150Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f6afa6590463bd975b8a9ab4e6dee4b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.154Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-827ba728d70d0e4d608d9223: start","job_status":"start","queue_duration_s":0.882381,"scheduling_latency_s":0.882394} gitlab # [ 173.369638] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.165Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"827ba728d70d0e4d608d9223","created_at":"2026-09-23T19:20:05.150Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5f6afa6590463bd975b8a9ab4e6dee4b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.154Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-827ba728d70d0e4d608d9223: done: 0.12926 sec","job_status":"done","queue_duration_s":0.882381,"scheduling_latency_s":0.882394,"redis_calls":11,"redis_duration_s":0.00164,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000252,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000669,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000719,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.091265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000314565,"exclusive_lock_hold_duration_s":0.117846263,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.12926,"completed_at":"2026-09-23T19:20:06.165Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.042707,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.416954] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.166Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"3e4cec722a67dd237b827416","created_at":"2026-09-23T19:20:05.164Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e716dcf3a4299c60280abec42120653a","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.172Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-3e4cec722a67dd237b827416: start","job_status":"start","queue_duration_s":0.993942,"scheduling_latency_s":0.993957} gitlab # [ 173.417655] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.187Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"3e4cec722a67dd237b827416","created_at":"2026-09-23T19:20:05.164Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e716dcf3a4299c60280abec42120653a","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.172Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-3e4cec722a67dd237b827416: done: 0.020481 sec","job_status":"done","queue_duration_s":0.993942,"scheduling_latency_s":0.993957,"redis_calls":10,"redis_duration_s":0.00134,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000273,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000274,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000205892,"duration_s":0.020481,"completed_at":"2026-09-23T19:20:06.187Z","load_balancing_strategy":"primary","db_duration_s":0.00614,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.476483] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.188Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"7f937c52d5aafda7bc44accd","created_at":"2026-09-23T19:20:05.191Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37458abb29ccc42ebc02521b52154d28","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-23T19:20:05.195Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-7f937c52d5aafda7bc44accd: start","job_status":"start","queue_duration_s":0.993033,"scheduling_latency_s":0.993046} gitlab # [ 173.477105] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.192Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 173.477733] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.195Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"7f937c52d5aafda7bc44accd","created_at":"2026-09-23T19:20:05.191Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37458abb29ccc42ebc02521b52154d28","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-23T19:20:05.195Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-7f937c52d5aafda7bc44accd: done: 0.007597 sec","job_status":"done","queue_duration_s":0.993033,"scheduling_latency_s":0.993046,"redis_calls":7,"redis_duration_s":0.0008979999999999999,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000623,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007597,"completed_at":"2026-09-23T19:20:06.195Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.522254] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.196Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"2443a4e9940cb7e685c22f8e","created_at":"2026-09-23T19:20:05.202Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"56cb204e447745e8a987de1a282dd0bc","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.209Z","job_size_bytes":3,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-2443a4e9940cb7e685c22f8e: start","job_status":"start","queue_duration_s":0.987076,"scheduling_latency_s":0.987089} gitlab # [ 173.523025] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.206Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"2443a4e9940cb7e685c22f8e","created_at":"2026-09-23T19:20:05.202Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"56cb204e447745e8a987de1a282dd0bc","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.209Z","job_size_bytes":3,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-2443a4e9940cb7e685c22f8e: done: 0.010431 sec","job_status":"done","queue_duration_s":0.987076,"scheduling_latency_s":0.987089,"redis_calls":10,"redis_duration_s":0.001264,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000769,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000206451,"duration_s":0.010431,"completed_at":"2026-09-23T19:20:06.206Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000255,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.564716] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.207Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ba8116392e1df247a5c2afec","created_at":"2026-09-23T19:20:05.217Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.219Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ba8116392e1df247a5c2afec: start","job_status":"start","queue_duration_s":0.987949,"scheduling_latency_s":0.987961} gitlab # [ 173.565592] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.260Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"ba8116392e1df247a5c2afec","created_at":"2026-09-23T19:20:05.217Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.219Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-ba8116392e1df247a5c2afec: done: 0.052361 sec","job_status":"done","queue_duration_s":0.987949,"scheduling_latency_s":0.987961,"redis_calls":60,"redis_duration_s":0.00807,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000448,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.007255,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000367,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052361,"completed_at":"2026-09-23T19:20:06.260Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.588994] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.261Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"398b33f2eae363578f40c12c","created_at":"2026-09-23T19:20:05.227Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d242b23175aeec81d71967a02309f7e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.258Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-398b33f2eae363578f40c12c: start","job_status":"start","queue_duration_s":1.002517,"scheduling_latency_s":1.002538} gitlab # [ 173.602201] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.269Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"398b33f2eae363578f40c12c","created_at":"2026-09-23T19:20:05.227Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d242b23175aeec81d71967a02309f7e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.258Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-398b33f2eae363578f40c12c: done: 0.007976 sec","job_status":"done","queue_duration_s":1.002517,"scheduling_latency_s":1.002538,"redis_calls":7,"redis_duration_s":0.0008359999999999999,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000606,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00023,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004917,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.007976,"completed_at":"2026-09-23T19:20:06.269Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.646758] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.269Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d8b9933cbb4516ea66751777","created_at":"2026-09-23T19:20:05.272Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0ceb73985963e22a108f7a5d19fc7d10","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.275Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d8b9933cbb4516ea66751777: start","job_status":"start","queue_duration_s":0.99433,"scheduling_latency_s":0.994342} gitlab # [ 173.685811] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.281Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d8b9933cbb4516ea66751777","created_at":"2026-09-23T19:20:05.272Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0ceb73985963e22a108f7a5d19fc7d10","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.275Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d8b9933cbb4516ea66751777: done: 0.011485 sec","job_status":"done","queue_duration_s":0.99433,"scheduling_latency_s":0.994342,"redis_calls":7,"redis_duration_s":0.000893,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000657,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000236,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011485,"completed_at":"2026-09-23T19:20:06.281Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002128,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.686945] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.282Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"ca1d82660513833a6ab5bf08","created_at":"2026-09-23T19:20:05.283Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6478d78bb0bf5a874ec7a4c93e366c0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.285Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-ca1d82660513833a6ab5bf08: start","job_status":"start","queue_duration_s":0.996428,"scheduling_latency_s":0.996442} gitlab # [ 173.698799] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.291Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"ca1d82660513833a6ab5bf08","created_at":"2026-09-23T19:20:05.283Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d6478d78bb0bf5a874ec7a4c93e366c0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.285Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-ca1d82660513833a6ab5bf08: done: 0.009471 sec","job_status":"done","queue_duration_s":0.996428,"scheduling_latency_s":0.996442,"redis_calls":9,"redis_duration_s":0.0012239999999999998,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000681,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000543,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000308698,"exclusive_lock_hold_duration_s":0.001681499,"duration_s":0.009471,"completed_at":"2026-09-23T19:20:06.291Z","load_balancing_strategy":"primary","db_duration_s":0.000394,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.743210] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.292Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::PublishEventWorker","jid":"bef1140e9856d1a6c9af2a8e","created_at":"2026-09-23T19:20:05.296Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3594ecd2ee20b6c580a4e80e975ed0d9","meta.root_caller_id":"Cronjob","meta.feature_category":"release_orchestration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9f5dbd6b3e0f8a09b95f6e4bc38c93c1137695f37704af6aead7c9933a73fb55","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.303Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::PublishEventWorker JID-bef1140e9856d1a6c9af2a8e: start","job_status":"start","queue_duration_s":0.988745,"scheduling_latency_s":0.988758} gitlab # [ 173.745758] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::PublishEventWorker","jid":"bef1140e9856d1a6c9af2a8e","created_at":"2026-09-23T19:20:05.296Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3594ecd2ee20b6c580a4e80e975ed0d9","meta.root_caller_id":"Cronjob","meta.feature_category":"release_orchestration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9f5dbd6b3e0f8a09b95f6e4bc38c93c1137695f37704af6aead7c9933a73fb55","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.303Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::PublishEventWorker JID-bef1140e9856d1a6c9af2a8e: done: 0.026071 sec","job_status":"done","queue_duration_s":0.988745,"scheduling_latency_s":0.988758,"redis_calls":10,"redis_duration_s":0.001312,"redis_read_bytes":9,"redis_write_bytes":1039,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000243,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000763,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000210083,"duration_s":0.026071,"completed_at":"2026-09-23T19:20:06.318Z","load_balancing_strategy":"primary","db_duration_s":0.007071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.785056] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.319Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"b06b032ba15a7f8cd55c633f","created_at":"2026-09-23T19:20:05.310Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3d88ff8bdd0fae2cdf627e7bc630fde","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.318Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-b06b032ba15a7f8cd55c633f: start","job_status":"start","queue_duration_s":1.001288,"scheduling_latency_s":1.001301} gitlab # [ 173.793208] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.331Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"b06b032ba15a7f8cd55c633f","created_at":"2026-09-23T19:20:05.310Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3d88ff8bdd0fae2cdf627e7bc630fde","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.318Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-b06b032ba15a7f8cd55c633f: done: 0.012506 sec","job_status":"done","queue_duration_s":1.001288,"scheduling_latency_s":1.001301,"redis_calls":10,"redis_duration_s":0.001256,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000261,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000751,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000244,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000209244,"duration_s":0.012506,"completed_at":"2026-09-23T19:20:06.331Z","load_balancing_strategy":"primary","db_duration_s":0.000254,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.833196] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.332Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8ddff222ad36d3bb52ee28d2","created_at":"2026-09-23T19:20:05.325Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ed77f3d612626943097c2052a38e16e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.329Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8ddff222ad36d3bb52ee28d2: start","job_status":"start","queue_duration_s":1.003663,"scheduling_latency_s":1.003675} gitlab # [ 173.842184] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.358Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8ddff222ad36d3bb52ee28d2","created_at":"2026-09-23T19:20:05.325Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5ed77f3d612626943097c2052a38e16e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.329Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8ddff222ad36d3bb52ee28d2: done: 0.025492 sec","job_status":"done","queue_duration_s":1.003663,"scheduling_latency_s":1.003675,"redis_calls":9,"redis_duration_s":0.0012330000000000002,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000604,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000359543,"exclusive_lock_hold_duration_s":0.009730287,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.025492,"completed_at":"2026-09-23T19:20:06.358Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012928,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.901873] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.359Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"6df34778774c1110382b23b6","created_at":"2026-09-23T19:20:05.337Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7bd7759e831c34fc9606f39b693df2a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.344Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-6df34778774c1110382b23b6: start","job_status":"start","queue_duration_s":1.014975,"scheduling_latency_s":1.014988} gitlab # [ 173.902738] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.369Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"6df34778774c1110382b23b6","created_at":"2026-09-23T19:20:05.337Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7bd7759e831c34fc9606f39b693df2a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.344Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-6df34778774c1110382b23b6: done: 0.010224 sec","job_status":"done","queue_duration_s":1.014975,"scheduling_latency_s":1.014988,"redis_calls":7,"redis_duration_s":0.000994,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000317,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.010224,"completed_at":"2026-09-23T19:20:06.369Z","load_balancing_strategy":"primary","db_duration_s":0.001896,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.943785] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.370Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"759b0630ac1f56b5ee61628f","created_at":"2026-09-23T19:20:05.355Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1803667d49d36486108d120e42e50205","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.367Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-759b0630ac1f56b5ee61628f: start","job_status":"start","queue_duration_s":1.002511,"scheduling_latency_s":1.002524} gitlab # [ 173.944644] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.388Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"759b0630ac1f56b5ee61628f","created_at":"2026-09-23T19:20:05.355Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-20d249c2c594456f","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1803667d49d36486108d120e42e50205","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.367Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-759b0630ac1f56b5ee61628f: done: 0.018481 sec","job_status":"done","queue_duration_s":1.002511,"scheduling_latency_s":1.002524,"redis_calls":20,"redis_duration_s":0.002322,"redis_read_bytes":23,"redis_write_bytes":4315,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.000731,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.000364,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2020,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.000967,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012333,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000200025,"duration_s":0.018481,"completed_at":"2026-09-23T19:20:06.388Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000879,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 173.983980] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.389Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c368d1a5b45b11205bbfeeda","created_at":"2026-09-23T19:20:05.522Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-907a8aa9b0c04f1b","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"833faa455c38103f7a85b661718acaab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.530Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c368d1a5b45b11205bbfeeda: start","job_status":"start","queue_duration_s":0.858564,"scheduling_latency_s":0.858578} gitlab # [ 174.022953] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.398Z","message":"No existing merge request to be cleaned up."} gitlab # [ 174.023725] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.403Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c368d1a5b45b11205bbfeeda","created_at":"2026-09-23T19:20:05.522Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-907a8aa9b0c04f1b","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"833faa455c38103f7a85b661718acaab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.530Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c368d1a5b45b11205bbfeeda: done: 0.014372 sec","job_status":"done","queue_duration_s":0.858564,"scheduling_latency_s":0.858578,"redis_calls":10,"redis_duration_s":0.0013779999999999999,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000778,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014372,"completed_at":"2026-09-23T19:20:06.403Z","load_balancing_strategy":"primary","db_duration_s":0.003856,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.024687] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.405Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5a3929538fdbec01d5ef77a1","created_at":"2026-09-23T19:20:05.522Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-907a8aa9b0c04f1b","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"833faa455c38103f7a85b661718acaab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.530Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5a3929538fdbec01d5ef77a1: start","job_status":"start","queue_duration_s":0.873983,"scheduling_latency_s":0.873997} gitlab # [ 174.036630] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.411Z","message":"No existing merge request to be cleaned up."} gitlab # [ 174.040240] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.416Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5a3929538fdbec01d5ef77a1","created_at":"2026-09-23T19:20:05.522Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-907a8aa9b0c04f1b","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"833faa455c38103f7a85b661718acaab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.530Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5a3929538fdbec01d5ef77a1: done: 0.011118 sec","job_status":"done","queue_duration_s":0.873983,"scheduling_latency_s":0.873997,"redis_calls":10,"redis_duration_s":0.0014520000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007498,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011118,"completed_at":"2026-09-23T19:20:06.416Z","load_balancing_strategy":"primary","db_duration_s":0.000964,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.078780] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.417Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c0a3a7ff4e02ccda8890f899","created_at":"2026-09-23T19:20:05.522Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-907a8aa9b0c04f1b","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"833faa455c38103f7a85b661718acaab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.530Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c0a3a7ff4e02ccda8890f899: start","job_status":"start","queue_duration_s":0.886077,"scheduling_latency_s":0.88609} gitlab # [ 174.080580] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.423Z","message":"No existing merge request to be cleaned up."} gitlab # [ 174.082705] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.427Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c0a3a7ff4e02ccda8890f899","created_at":"2026-09-23T19:20:05.522Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-907a8aa9b0c04f1b","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"833faa455c38103f7a85b661718acaab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.530Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c0a3a7ff4e02ccda8890f899: done: 0.010616 sec","job_status":"done","queue_duration_s":0.886077,"scheduling_latency_s":0.88609,"redis_calls":10,"redis_duration_s":0.001184,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000489,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000695,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010616,"completed_at":"2026-09-23T19:20:06.427Z","load_balancing_strategy":"primary","db_duration_s":0.000889,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.130565] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.428Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"94dc03afc964166b039cf55c","created_at":"2026-09-23T19:20:05.522Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-907a8aa9b0c04f1b","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"833faa455c38103f7a85b661718acaab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.530Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-94dc03afc964166b039cf55c: start","job_status":"start","queue_duration_s":0.897769,"scheduling_latency_s":0.897782} gitlab # [ 174.131118] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.435Z","message":"No existing merge request to be cleaned up."} gitlab # [ 174.131648] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.440Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"94dc03afc964166b039cf55c","created_at":"2026-09-23T19:20:05.522Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-907a8aa9b0c04f1b","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"833faa455c38103f7a85b661718acaab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.530Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-94dc03afc964166b039cf55c: done: 0.01127 sec","job_status":"done","queue_duration_s":0.897769,"scheduling_latency_s":0.897782,"redis_calls":10,"redis_duration_s":0.001179,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000485,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000694,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01127,"completed_at":"2026-09-23T19:20:06.440Z","load_balancing_strategy":"primary","db_duration_s":0.000902,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.185196] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.440Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"repository_check","args":["default"],"class":"RepositoryCheck::BatchWorker","jid":"dde2805db896bae472833f14","created_at":"2026-09-23T19:20:05.825Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-83e0186669374c5c","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"RepositoryCheck::DispatchWorker","correlation_id":"89d2d5558af3c5678c4e711eabcca62a","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b22e99a0acd71e0cf2962b3ba955d6f34621628300baf74d4c861b087bc17605","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.828Z","job_size_bytes":11,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryCheck::BatchWorker JID-dde2805db896bae472833f14: start","job_status":"start","queue_duration_s":0.612493,"scheduling_latency_s":0.612506} gitlab # [ 174.185947] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.465Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"repository_check","args":["default"],"class":"RepositoryCheck::BatchWorker","jid":"dde2805db896bae472833f14","created_at":"2026-09-23T19:20:05.825Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-83e0186669374c5c","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"RepositoryCheck::DispatchWorker","correlation_id":"89d2d5558af3c5678c4e711eabcca62a","meta.root_caller_id":"Cronjob","meta.feature_category":"source_code_management","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b22e99a0acd71e0cf2962b3ba955d6f34621628300baf74d4c861b087bc17605","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:05.828Z","job_size_bytes":11,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"RepositoryCheck::BatchWorker JID-dde2805db896bae472833f14: done: 0.024335 sec","job_status":"done","queue_duration_s":0.612493,"scheduling_latency_s":0.612506,"redis_calls":13,"redis_duration_s":0.0018399999999999998,"redis_read_bytes":12,"redis_write_bytes":1387,"redis_cache_calls":1,"redis_cache_duration_s":0.000178,"redis_cache_read_bytes":1,"redis_cache_write_bytes":37,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000269,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000752,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000641,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":513,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.006029257,"duration_s":0.024335,"completed_at":"2026-09-23T19:20:06.465Z","load_balancing_strategy":"primary","db_duration_s":0.003071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.228589] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.466Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16c23055bcc852917f748b44","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16c23055bcc852917f748b44: start","job_status":"start","queue_duration_s":0.211531,"scheduling_latency_s":0.211544} gitlab # [ 174.229420] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16c23055bcc852917f748b44","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16c23055bcc852917f748b44: done: 0.010959 sec","job_status":"done","queue_duration_s":0.211531,"scheduling_latency_s":0.211544,"redis_calls":13,"redis_duration_s":0.001451,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000482,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010959,"completed_at":"2026-09-23T19:20:06.477Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.285624] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"998eb526f70945005fcf9d7c","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-998eb526f70945005fcf9d7c: start","job_status":"start","queue_duration_s":0.223569,"scheduling_latency_s":0.223584} gitlab # [ 174.286500] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.489Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"998eb526f70945005fcf9d7c","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-998eb526f70945005fcf9d7c: done: 0.010851 sec","job_status":"done","queue_duration_s":0.223569,"scheduling_latency_s":0.223584,"redis_calls":13,"redis_duration_s":0.001704,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000453,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001251,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010851,"completed_at":"2026-09-23T19:20:06.489Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.328438] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acd7b1cfd772731dbf9092ac","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acd7b1cfd772731dbf9092ac: start","job_status":"start","queue_duration_s":0.235719,"scheduling_latency_s":0.235732} gitlab # [ 174.329069] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"acd7b1cfd772731dbf9092ac","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-acd7b1cfd772731dbf9092ac: done: 0.011979 sec","job_status":"done","queue_duration_s":0.235719,"scheduling_latency_s":0.235732,"redis_calls":13,"redis_duration_s":0.001646,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000555,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011979,"completed_at":"2026-09-23T19:20:06.502Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.372672] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"626708d37c219a9b4deb608e","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-626708d37c219a9b4deb608e: start","job_status":"start","queue_duration_s":0.248639,"scheduling_latency_s":0.248653} gitlab # [ 174.373676] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.514Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"626708d37c219a9b4deb608e","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-626708d37c219a9b4deb608e: done: 0.010782 sec","job_status":"done","queue_duration_s":0.248639,"scheduling_latency_s":0.248653,"redis_calls":13,"redis_duration_s":0.0016779999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000674,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010782,"completed_at":"2026-09-23T19:20:06.514Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.394600] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.515Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e33ea2d58a3f909afd947763","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e33ea2d58a3f909afd947763: start","job_status":"start","queue_duration_s":0.260216,"scheduling_latency_s":0.260229} gitlab # [ 174.405772] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e33ea2d58a3f909afd947763","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e33ea2d58a3f909afd947763: done: 0.009122 sec","job_status":"done","queue_duration_s":0.260216,"scheduling_latency_s":0.260229,"redis_calls":13,"redis_duration_s":0.0013909999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000452,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009122,"completed_at":"2026-09-23T19:20:06.524Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.461922] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63ffeec9a6e341e2bf4a691a","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63ffeec9a6e341e2bf4a691a: start","job_status":"start","queue_duration_s":0.270177,"scheduling_latency_s":0.27019} gitlab # [ 174.462614] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.533Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63ffeec9a6e341e2bf4a691a","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63ffeec9a6e341e2bf4a691a: done: 0.008649 sec","job_status":"done","queue_duration_s":0.270177,"scheduling_latency_s":0.27019,"redis_calls":13,"redis_duration_s":0.001391,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00047,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008649,"completed_at":"2026-09-23T19:20:06.533Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.511517] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88875cb85f63b1786017464c","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88875cb85f63b1786017464c: start","job_status":"start","queue_duration_s":0.279626,"scheduling_latency_s":0.279639} gitlab # [ 174.512150] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.543Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88875cb85f63b1786017464c","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88875cb85f63b1786017464c: done: 0.008899 sec","job_status":"done","queue_duration_s":0.279626,"scheduling_latency_s":0.279639,"redis_calls":13,"redis_duration_s":0.001557,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001029,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005825,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008899,"completed_at":"2026-09-23T19:20:06.543Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.551663] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.544Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1201358789803cff45acb8fb","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1201358789803cff45acb8fb: start","job_status":"start","queue_duration_s":0.289366,"scheduling_latency_s":0.289378} gitlab # [ 174.552601] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.553Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1201358789803cff45acb8fb","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1201358789803cff45acb8fb: done: 0.008717 sec","job_status":"done","queue_duration_s":0.289366,"scheduling_latency_s":0.289378,"redis_calls":13,"redis_duration_s":0.001408,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000462,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008717,"completed_at":"2026-09-23T19:20:06.553Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.613598] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.553Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d7b12a7e1c7c7d86b501dd6","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d7b12a7e1c7c7d86b501dd6: start","job_status":"start","queue_duration_s":0.298859,"scheduling_latency_s":0.298896} gitlab # [ 174.614424] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d7b12a7e1c7c7d86b501dd6","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d7b12a7e1c7c7d86b501dd6: done: 0.008766 sec","job_status":"done","queue_duration_s":0.298859,"scheduling_latency_s":0.298896,"redis_calls":13,"redis_duration_s":0.0014589999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000463,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008766,"completed_at":"2026-09-23T19:20:06.562Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.654731] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b78575692142131771816fcf","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b78575692142131771816fcf: start","job_status":"start","queue_duration_s":0.308413,"scheduling_latency_s":0.308435} gitlab # [ 174.655612] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b78575692142131771816fcf","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b78575692142131771816fcf: done: 0.008612 sec","job_status":"done","queue_duration_s":0.308413,"scheduling_latency_s":0.308435,"redis_calls":13,"redis_duration_s":0.0014160000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000451,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008612,"completed_at":"2026-09-23T19:20:06.572Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.669619] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"554b6c3d98228d400a5d9242","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-554b6c3d98228d400a5d9242: start","job_status":"start","queue_duration_s":0.31779,"scheduling_latency_s":0.317802} gitlab # [ 174.712765] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"554b6c3d98228d400a5d9242","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-554b6c3d98228d400a5d9242: done: 0.008685 sec","job_status":"done","queue_duration_s":0.31779,"scheduling_latency_s":0.317802,"redis_calls":13,"redis_duration_s":0.0014060000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000463,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005794,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008685,"completed_at":"2026-09-23T19:20:06.581Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.715060] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c23be5fda5a6663101dc86d","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c23be5fda5a6663101dc86d: start","job_status":"start","queue_duration_s":0.32727,"scheduling_latency_s":0.327283} gitlab # [ 174.728843] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c23be5fda5a6663101dc86d","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c23be5fda5a6663101dc86d: done: 0.008825 sec","job_status":"done","queue_duration_s":0.32727,"scheduling_latency_s":0.327283,"redis_calls":13,"redis_duration_s":0.0015769999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008825,"completed_at":"2026-09-23T19:20:06.591Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.775019] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c46bd5eac6fee3f0cb1244c","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c46bd5eac6fee3f0cb1244c: start","job_status":"start","queue_duration_s":0.337002,"scheduling_latency_s":0.337015} gitlab # [ 174.775744] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5c46bd5eac6fee3f0cb1244c","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5c46bd5eac6fee3f0cb1244c: done: 0.008845 sec","job_status":"done","queue_duration_s":0.337002,"scheduling_latency_s":0.337015,"redis_calls":13,"redis_duration_s":0.001541,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008845,"completed_at":"2026-09-23T19:20:06.600Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.821213] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"924f5fc6361a9da1bee1b8e2","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-924f5fc6361a9da1bee1b8e2: start","job_status":"start","queue_duration_s":0.346669,"scheduling_latency_s":0.346682} gitlab # [ 174.822052] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.610Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"924f5fc6361a9da1bee1b8e2","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-924f5fc6361a9da1bee1b8e2: done: 0.008842 sec","job_status":"done","queue_duration_s":0.346669,"scheduling_latency_s":0.346682,"redis_calls":13,"redis_duration_s":0.001588,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000483,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008842,"completed_at":"2026-09-23T19:20:06.610Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.874612] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86270f383fae28cfe1d310f0","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86270f383fae28cfe1d310f0: start","job_status":"start","queue_duration_s":0.356312,"scheduling_latency_s":0.356325} gitlab # [ 174.875523] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.622Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86270f383fae28cfe1d310f0","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86270f383fae28cfe1d310f0: done: 0.010829 sec","job_status":"done","queue_duration_s":0.356312,"scheduling_latency_s":0.356325,"redis_calls":13,"redis_duration_s":0.001617,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000655,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010829,"completed_at":"2026-09-23T19:20:06.622Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.898703] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.623Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59af3e3bfb04cc97f37af0c4","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59af3e3bfb04cc97f37af0c4: start","job_status":"start","queue_duration_s":0.368356,"scheduling_latency_s":0.368377} gitlab # [ 174.909757] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.633Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59af3e3bfb04cc97f37af0c4","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59af3e3bfb04cc97f37af0c4: done: 0.010083 sec","job_status":"done","queue_duration_s":0.368356,"scheduling_latency_s":0.368377,"redis_calls":13,"redis_duration_s":0.00147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000481,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010083,"completed_at":"2026-09-23T19:20:06.633Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.955769] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c89ed64c1b7fc6e6f9f52e0","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c89ed64c1b7fc6e6f9f52e0: start","job_status":"start","queue_duration_s":0.379216,"scheduling_latency_s":0.379228} gitlab # [ 174.956665] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c89ed64c1b7fc6e6f9f52e0","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c89ed64c1b7fc6e6f9f52e0: done: 0.009572 sec","job_status":"done","queue_duration_s":0.379216,"scheduling_latency_s":0.379228,"redis_calls":13,"redis_duration_s":0.001544,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006404,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009572,"completed_at":"2026-09-23T19:20:06.643Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 174.996815] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65ed81e549df61a3ca7150e6","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65ed81e549df61a3ca7150e6: start","job_status":"start","queue_duration_s":0.389703,"scheduling_latency_s":0.389718} gitlab # [ 175.000729] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.653Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65ed81e549df61a3ca7150e6","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65ed81e549df61a3ca7150e6: done: 0.00915 sec","job_status":"done","queue_duration_s":0.389703,"scheduling_latency_s":0.389718,"redis_calls":13,"redis_duration_s":0.001529,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00915,"completed_at":"2026-09-23T19:20:06.653Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.052572] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a89f317aa4bfa3369cc56367","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a89f317aa4bfa3369cc56367: start","job_status":"start","queue_duration_s":0.399668,"scheduling_latency_s":0.399681} gitlab # [ 175.053456] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a89f317aa4bfa3369cc56367","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a89f317aa4bfa3369cc56367: done: 0.00869 sec","job_status":"done","queue_duration_s":0.399668,"scheduling_latency_s":0.399681,"redis_calls":13,"redis_duration_s":0.001421,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000458,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00869,"completed_at":"2026-09-23T19:20:06.663Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.095721] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7ed5059d408a340451883b8","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7ed5059d408a340451883b8: start","job_status":"start","queue_duration_s":0.409163,"scheduling_latency_s":0.409176} gitlab # [ 175.096645] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.672Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7ed5059d408a340451883b8","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7ed5059d408a340451883b8: done: 0.008607 sec","job_status":"done","queue_duration_s":0.409163,"scheduling_latency_s":0.409176,"redis_calls":13,"redis_duration_s":0.0014140000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000462,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008607,"completed_at":"2026-09-23T19:20:06.672Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.147884] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5107fce84231dc6ef255629a","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5107fce84231dc6ef255629a: start","job_status":"start","queue_duration_s":0.418564,"scheduling_latency_s":0.418576} gitlab # [ 175.148759] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.682Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5107fce84231dc6ef255629a","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5107fce84231dc6ef255629a: done: 0.008609 sec","job_status":"done","queue_duration_s":0.418564,"scheduling_latency_s":0.418576,"redis_calls":13,"redis_duration_s":0.001429,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000483,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008609,"completed_at":"2026-09-23T19:20:06.682Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.196149] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.683Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"323de31f57bd70c59ebee312","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-323de31f57bd70c59ebee312: start","job_status":"start","queue_duration_s":0.427959,"scheduling_latency_s":0.427971} gitlab # [ 175.196798] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"323de31f57bd70c59ebee312","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-323de31f57bd70c59ebee312: done: 0.008709 sec","job_status":"done","queue_duration_s":0.427959,"scheduling_latency_s":0.427971,"redis_calls":13,"redis_duration_s":0.0015119999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000494,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001018,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005711,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008709,"completed_at":"2026-09-23T19:20:06.691Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.252291] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a4c5c1a87e0cf9a0bdfe035","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a4c5c1a87e0cf9a0bdfe035: start","job_status":"start","queue_duration_s":0.437537,"scheduling_latency_s":0.43755} gitlab # [ 175.253140] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.701Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a4c5c1a87e0cf9a0bdfe035","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a4c5c1a87e0cf9a0bdfe035: done: 0.00889 sec","job_status":"done","queue_duration_s":0.437537,"scheduling_latency_s":0.43755,"redis_calls":13,"redis_duration_s":0.001547,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00567,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00889,"completed_at":"2026-09-23T19:20:06.701Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.264607] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7c7a94bcacd03616f067c04","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7c7a94bcacd03616f067c04: start","job_status":"start","queue_duration_s":0.44725,"scheduling_latency_s":0.447264} gitlab # [ 175.275813] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7c7a94bcacd03616f067c04","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7c7a94bcacd03616f067c04: done: 0.008888 sec","job_status":"done","queue_duration_s":0.44725,"scheduling_latency_s":0.447264,"redis_calls":13,"redis_duration_s":0.001537,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008888,"completed_at":"2026-09-23T19:20:06.711Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.316242] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.712Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"920ecee8e3818013a707eb2f","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-920ecee8e3818013a707eb2f: start","job_status":"start","queue_duration_s":0.456989,"scheduling_latency_s":0.457001} gitlab # [ 175.359779] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"920ecee8e3818013a707eb2f","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-920ecee8e3818013a707eb2f: done: 0.008938 sec","job_status":"done","queue_duration_s":0.456989,"scheduling_latency_s":0.457001,"redis_calls":13,"redis_duration_s":0.001494,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008938,"completed_at":"2026-09-23T19:20:06.720Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.360891] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6195bf6f29a8408fc313ca56","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6195bf6f29a8408fc313ca56: start","job_status":"start","queue_duration_s":0.466696,"scheduling_latency_s":0.466709} gitlab # [ 175.402349] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6195bf6f29a8408fc313ca56","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6195bf6f29a8408fc313ca56: done: 0.008651 sec","job_status":"done","queue_duration_s":0.466696,"scheduling_latency_s":0.466709,"redis_calls":13,"redis_duration_s":0.001384,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000451,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008651,"completed_at":"2026-09-23T19:20:06.730Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.403711] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffbef2a4a786fa67090e99b8","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffbef2a4a786fa67090e99b8: start","job_status":"start","queue_duration_s":0.476157,"scheduling_latency_s":0.476169} gitlab # [ 175.445531] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.739Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ffbef2a4a786fa67090e99b8","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ffbef2a4a786fa67090e99b8: done: 0.008675 sec","job_status":"done","queue_duration_s":0.476157,"scheduling_latency_s":0.476169,"redis_calls":13,"redis_duration_s":0.001416,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000463,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008675,"completed_at":"2026-09-23T19:20:06.739Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.447729] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba7faf401926d696adbb9f75","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba7faf401926d696adbb9f75: start","job_status":"start","queue_duration_s":0.485616,"scheduling_latency_s":0.485629} gitlab # [ 175.458771] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba7faf401926d696adbb9f75","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba7faf401926d696adbb9f75: done: 0.008654 sec","job_status":"done","queue_duration_s":0.485616,"scheduling_latency_s":0.485629,"redis_calls":13,"redis_duration_s":0.001397,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000453,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008654,"completed_at":"2026-09-23T19:20:06.749Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.499077] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb5f5357509bee4dfdf638bb","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb5f5357509bee4dfdf638bb: start","job_status":"start","queue_duration_s":0.495141,"scheduling_latency_s":0.495154} gitlab # [ 175.504287] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb5f5357509bee4dfdf638bb","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb5f5357509bee4dfdf638bb: done: 0.00888 sec","job_status":"done","queue_duration_s":0.495141,"scheduling_latency_s":0.495154,"redis_calls":13,"redis_duration_s":0.0015899999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001063,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00888,"completed_at":"2026-09-23T19:20:06.759Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.539544] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42bdcaffef7ab9fd44b5f2fc","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42bdcaffef7ab9fd44b5f2fc: start","job_status":"start","queue_duration_s":0.504863,"scheduling_latency_s":0.504899} gitlab # [ 175.550797] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"42bdcaffef7ab9fd44b5f2fc","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-42bdcaffef7ab9fd44b5f2fc: done: 0.008689 sec","job_status":"done","queue_duration_s":0.504863,"scheduling_latency_s":0.504899,"redis_calls":13,"redis_duration_s":0.001439,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000471,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008689,"completed_at":"2026-09-23T19:20:06.768Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.592503] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a856fc68d76b30eaaca3f16c","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a856fc68d76b30eaaca3f16c: start","job_status":"start","queue_duration_s":0.514336,"scheduling_latency_s":0.514348} gitlab # [ 175.596089] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a856fc68d76b30eaaca3f16c","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a856fc68d76b30eaaca3f16c: done: 0.008637 sec","job_status":"done","queue_duration_s":0.514336,"scheduling_latency_s":0.514348,"redis_calls":13,"redis_duration_s":0.001433,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000477,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005765,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008637,"completed_at":"2026-09-23T19:20:06.778Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.655632] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95b4a47453330c888d77f815","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95b4a47453330c888d77f815: start","job_status":"start","queue_duration_s":0.523777,"scheduling_latency_s":0.52379} gitlab # [ 175.656417] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95b4a47453330c888d77f815","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95b4a47453330c888d77f815: done: 0.008616 sec","job_status":"done","queue_duration_s":0.523777,"scheduling_latency_s":0.52379,"redis_calls":13,"redis_duration_s":0.001396,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000458,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008616,"completed_at":"2026-09-23T19:20:06.787Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.702850] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4db0976ebe4a947ae56c02be","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4db0976ebe4a947ae56c02be: start","job_status":"start","queue_duration_s":0.533187,"scheduling_latency_s":0.5332} gitlab # [ 175.703682] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4db0976ebe4a947ae56c02be","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4db0976ebe4a947ae56c02be: done: 0.008814 sec","job_status":"done","queue_duration_s":0.533187,"scheduling_latency_s":0.5332,"redis_calls":13,"redis_duration_s":0.00164,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008814,"completed_at":"2026-09-23T19:20:06.797Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.721777] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"857f7ae9a13375a49154bb80","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-857f7ae9a13375a49154bb80: start","job_status":"start","queue_duration_s":0.542803,"scheduling_latency_s":0.542815} gitlab # [ 175.761654] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.806Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"857f7ae9a13375a49154bb80","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-857f7ae9a13375a49154bb80: done: 0.008658 sec","job_status":"done","queue_duration_s":0.542803,"scheduling_latency_s":0.542815,"redis_calls":13,"redis_duration_s":0.001425,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000465,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008658,"completed_at":"2026-09-23T19:20:06.806Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.800661] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"971c50dfa9acadd6e13b6fcf","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-971c50dfa9acadd6e13b6fcf: start","job_status":"start","queue_duration_s":0.552256,"scheduling_latency_s":0.552269} gitlab # [ 175.801545] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"971c50dfa9acadd6e13b6fcf","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-971c50dfa9acadd6e13b6fcf: done: 0.008836 sec","job_status":"done","queue_duration_s":0.552256,"scheduling_latency_s":0.552269,"redis_calls":13,"redis_duration_s":0.001594,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008836,"completed_at":"2026-09-23T19:20:06.816Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.840103] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1325713ad0d8c498f7ffa0ea","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1325713ad0d8c498f7ffa0ea: start","job_status":"start","queue_duration_s":0.561864,"scheduling_latency_s":0.561898} gitlab # [ 175.840795] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.825Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1325713ad0d8c498f7ffa0ea","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1325713ad0d8c498f7ffa0ea: done: 0.008706 sec","job_status":"done","queue_duration_s":0.561864,"scheduling_latency_s":0.561898,"redis_calls":13,"redis_duration_s":0.001417,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000462,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008706,"completed_at":"2026-09-23T19:20:06.825Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.857149] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"295592edd51f3d601c3099f3","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-295592edd51f3d601c3099f3: start","job_status":"start","queue_duration_s":0.571442,"scheduling_latency_s":0.571455} gitlab # [ 175.895268] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"295592edd51f3d601c3099f3","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-295592edd51f3d601c3099f3: done: 0.008658 sec","job_status":"done","queue_duration_s":0.571442,"scheduling_latency_s":0.571455,"redis_calls":13,"redis_duration_s":0.00149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000475,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008658,"completed_at":"2026-09-23T19:20:06.835Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.902609] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"442e6dc83809760d321dc0a4","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-442e6dc83809760d321dc0a4: start","job_status":"start","queue_duration_s":0.580894,"scheduling_latency_s":0.580907} gitlab # [ 175.913630] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"442e6dc83809760d321dc0a4","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-442e6dc83809760d321dc0a4: done: 0.008883 sec","job_status":"done","queue_duration_s":0.580894,"scheduling_latency_s":0.580907,"redis_calls":13,"redis_duration_s":0.0015509999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008883,"completed_at":"2026-09-23T19:20:06.844Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 175.975489] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bc5fa7e0f20bc8ed36c5695","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bc5fa7e0f20bc8ed36c5695: start","job_status":"start","queue_duration_s":0.590582,"scheduling_latency_s":0.590594} gitlab # [ 175.976279] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7bc5fa7e0f20bc8ed36c5695","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7bc5fa7e0f20bc8ed36c5695: done: 0.008854 sec","job_status":"done","queue_duration_s":0.590582,"scheduling_latency_s":0.590594,"redis_calls":13,"redis_duration_s":0.0015509999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001045,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008854,"completed_at":"2026-09-23T19:20:06.854Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.025240] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.855Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14c964e19b40e793f120fbab","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14c964e19b40e793f120fbab: start","job_status":"start","queue_duration_s":0.600248,"scheduling_latency_s":0.60026} gitlab # [ 176.026173] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14c964e19b40e793f120fbab","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14c964e19b40e793f120fbab: done: 0.008828 sec","job_status":"done","queue_duration_s":0.600248,"scheduling_latency_s":0.60026,"redis_calls":13,"redis_duration_s":0.0014990000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008828,"completed_at":"2026-09-23T19:20:06.864Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.071884] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c8d4fea5e34273f5cc7cfb4","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c8d4fea5e34273f5cc7cfb4: start","job_status":"start","queue_duration_s":0.609954,"scheduling_latency_s":0.609968} gitlab # [ 176.072668] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c8d4fea5e34273f5cc7cfb4","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c8d4fea5e34273f5cc7cfb4: done: 0.008834 sec","job_status":"done","queue_duration_s":0.609954,"scheduling_latency_s":0.609968,"redis_calls":13,"redis_duration_s":0.001516,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008834,"completed_at":"2026-09-23T19:20:06.873Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.086387] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fd71b7627e93ff6fbdf7cad","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fd71b7627e93ff6fbdf7cad: start","job_status":"start","queue_duration_s":0.619609,"scheduling_latency_s":0.619622} gitlab # [ 176.132528] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fd71b7627e93ff6fbdf7cad","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fd71b7627e93ff6fbdf7cad: done: 0.008875 sec","job_status":"done","queue_duration_s":0.619609,"scheduling_latency_s":0.619622,"redis_calls":13,"redis_duration_s":0.001531,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008875,"completed_at":"2026-09-23T19:20:06.883Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.133649] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df4285adacccc2ecb823b200","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df4285adacccc2ecb823b200: start","job_status":"start","queue_duration_s":0.629353,"scheduling_latency_s":0.629367} gitlab # [ 176.143072] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df4285adacccc2ecb823b200","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df4285adacccc2ecb823b200: done: 0.008975 sec","job_status":"done","queue_duration_s":0.629353,"scheduling_latency_s":0.629367,"redis_calls":13,"redis_duration_s":0.0014269999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000456,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008975,"completed_at":"2026-09-23T19:20:06.893Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.182115] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99e3ec9f15cc66c7cf21c693","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99e3ec9f15cc66c7cf21c693: start","job_status":"start","queue_duration_s":0.639327,"scheduling_latency_s":0.63934} gitlab # [ 176.188765] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99e3ec9f15cc66c7cf21c693","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99e3ec9f15cc66c7cf21c693: done: 0.009105 sec","job_status":"done","queue_duration_s":0.639327,"scheduling_latency_s":0.63934,"redis_calls":13,"redis_duration_s":0.001559,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000472,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006234,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009105,"completed_at":"2026-09-23T19:20:06.903Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.230819] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d66d2fcb2b02e54f092baaea","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d66d2fcb2b02e54f092baaea: start","job_status":"start","queue_duration_s":0.649384,"scheduling_latency_s":0.649397} gitlab # [ 176.233889] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.913Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d66d2fcb2b02e54f092baaea","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d66d2fcb2b02e54f092baaea: done: 0.009179 sec","job_status":"done","queue_duration_s":0.649384,"scheduling_latency_s":0.649397,"redis_calls":13,"redis_duration_s":0.001423,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000474,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009179,"completed_at":"2026-09-23T19:20:06.913Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.291682] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4c2e7cb620c9f8fc31936f4","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4c2e7cb620c9f8fc31936f4: start","job_status":"start","queue_duration_s":0.659378,"scheduling_latency_s":0.659391} gitlab # [ 176.292605] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4c2e7cb620c9f8fc31936f4","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4c2e7cb620c9f8fc31936f4: done: 0.009548 sec","job_status":"done","queue_duration_s":0.659378,"scheduling_latency_s":0.659391,"redis_calls":13,"redis_duration_s":0.001618,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006267,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009548,"completed_at":"2026-09-23T19:20:06.923Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.314709] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12b7e82ed4461c8a389aaee5","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12b7e82ed4461c8a389aaee5: start","job_status":"start","queue_duration_s":0.669725,"scheduling_latency_s":0.669738} gitlab # [ 176.325710] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12b7e82ed4461c8a389aaee5","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12b7e82ed4461c8a389aaee5: done: 0.009372 sec","job_status":"done","queue_duration_s":0.669725,"scheduling_latency_s":0.669738,"redis_calls":13,"redis_duration_s":0.00163,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000487,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001143,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009372,"completed_at":"2026-09-23T19:20:06.934Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.381165] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97d39a3347610d3314ab8a99","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97d39a3347610d3314ab8a99: start","job_status":"start","queue_duration_s":0.679909,"scheduling_latency_s":0.679921} gitlab # [ 176.381865] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.944Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97d39a3347610d3314ab8a99","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97d39a3347610d3314ab8a99: done: 0.009547 sec","job_status":"done","queue_duration_s":0.679909,"scheduling_latency_s":0.679921,"redis_calls":13,"redis_duration_s":0.001557,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000459,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009547,"completed_at":"2026-09-23T19:20:06.944Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.441192] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69c345e96c02ba5b4ca496d2","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69c345e96c02ba5b4ca496d2: start","job_status":"start","queue_duration_s":0.690231,"scheduling_latency_s":0.690244} gitlab # [ 176.441932] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69c345e96c02ba5b4ca496d2","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69c345e96c02ba5b4ca496d2: done: 0.009273 sec","job_status":"done","queue_duration_s":0.690231,"scheduling_latency_s":0.690244,"redis_calls":13,"redis_duration_s":0.001548,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000473,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009273,"completed_at":"2026-09-23T19:20:06.954Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.485214] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"537890d7e409c2f164d780ce","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-537890d7e409c2f164d780ce: start","job_status":"start","queue_duration_s":0.700309,"scheduling_latency_s":0.700322} gitlab # [ 176.485467] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"537890d7e409c2f164d780ce","created_at":"2026-09-23T19:20:06.213Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-2ea5474e603146f1","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0fe7baec06ca9189376c013e5b3aab59","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.255Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-537890d7e409c2f164d780ce: done: 0.009582 sec","job_status":"done","queue_duration_s":0.700309,"scheduling_latency_s":0.700322,"redis_calls":13,"redis_duration_s":0.0016849999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001132,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009582,"completed_at":"2026-09-23T19:20:06.964Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.526709] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.965Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"0b070286fc13b2aa9348dc6f","created_at":"2026-09-23T19:20:06.376Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-da8eb4ca7cfe4e66","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"1803667d49d36486108d120e42e50205","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.381Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-0b070286fc13b2aa9348dc6f: start","job_status":"start","queue_duration_s":0.584668,"scheduling_latency_s":0.584681} gitlab # [ 176.527106] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.985Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"0b070286fc13b2aa9348dc6f","created_at":"2026-09-23T19:20:06.376Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-da8eb4ca7cfe4e66","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"1803667d49d36486108d120e42e50205","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.381Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-0b070286fc13b2aa9348dc6f: done: 0.019311 sec","job_status":"done","queue_duration_s":0.584668,"scheduling_latency_s":0.584681,"redis_calls":10,"redis_duration_s":0.001409,"redis_read_bytes":9,"redis_write_bytes":1117,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000817,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000289,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":222,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000360381,"duration_s":0.019311,"completed_at":"2026-09-23T19:20:06.985Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005339,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 176.579276] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.986Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"d95e6467bfc85ee43255bdc6","created_at":"2026-09-23T19:20:06.381Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-da8eb4ca7cfe4e66","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"1803667d49d36486108d120e42e50205","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.384Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-d95e6467bfc85ee43255bdc6: start","job_status":"start","queue_duration_s":0.602145,"scheduling_latency_s":0.602157} gitlab # [ 176.580091] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:06.998Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"d95e6467bfc85ee43255bdc6","created_at":"2026-09-23T19:20:06.381Z","trace_propagation_headers":{"sentry-trace":"950f8a069a294920a794dd711ca0e968-da8eb4ca7cfe4e66","baggage":"sentry-trace_id=950f8a069a294920a794dd711ca0e968,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"1803667d49d36486108d120e42e50205","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:06.384Z","job_size_bytes":2,"pid":1621,"sidekiq_tid":"madd","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-d95e6467bfc85ee43255bdc6: done: 0.012735 sec","job_status":"done","queue_duration_s":0.602145,"scheduling_latency_s":0.602157,"redis_calls":10,"redis_duration_s":0.001701,"redis_read_bytes":9,"redis_write_bytes":1133,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000285,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001177,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000239,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000189969,"duration_s":0.012735,"completed_at":"2026-09-23T19:20:06.998Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000713,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 182.519572] postgres[2426]: [2426] ERROR: relation "table_without_models" does not exist gitlab # [ 182.520139] postgres[2426]: [2426] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 185.431876] runuser[2217]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 185.433682] setup-gitlab-start[2199]: + systemctl restart gitlab-sidekiq.service gitlab # [ 185.462633] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 185.464805] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:18.737Z","message":"Shutting down"} gitlab # [ 185.465188] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:18.737Z","message":"Scheduler exiting..."} gitlab # [ 185.471134] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:18.743Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 185.471374] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:18.744Z","message":"Scheduler exiting..."} gitlab # [ 185.979741] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:19.251Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 188.984629] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:22.256Z","message":"Bye!"} gitlab # [ 189.009468] sidekiq-cluster[1621]: {"severity":"INFO","time":"2026-09-23T19:20:22.282Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1621,"worker_id":"sidekiq_0","memwd_rss_bytes":955797504,"retry":0} gitlab # [ 189.992494] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 189.995226] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 189.997095] systemd[1]: gitlab-sidekiq.service: Consumed 57.001s CPU time over 1min 56.070s wall clock time, 956.9M memory peak, 4.4M read from disk. gitlab # [ 190.005767] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 190.009658] setup-gitlab-start[2199]: + mkdir -p /var/lib/secrets gitlab # [ 190.014776] setup-gitlab-start[2199]: + echo nixbot-test-token-of-20-chars gitlab # [ 190.016220] setup-gitlab-start[2199]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 190.022262] setup-gitlab-start[2199]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars gitlab # [ 190.492696] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:23 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 442 gitlab # [ 191.059241] gitaly[890]: time="2026-09-23T19:20:24.331Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:20:34.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.331" pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.068363] gitaly[890]: time="2026-09-23T19:20:24.331Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:20:34.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.331" grpc.time_ms=0.25 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.146148] gitaly[890]: time="2026-09-23T19:20:24.418Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:20:34.098" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.418" pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.147142] gitaly[890]: time="2026-09-23T19:20:24.419Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:20:34.098" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.418" grpc.time_ms=0.33 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.241363] gitaly[890]: time="2026-09-23T19:20:24.514Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 diskcache=a6a6e331-6dfa-4ffd-9d6d-88341caa3bfc grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-23T19:20:34.149" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.471" pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.259470] gitaly[890]: time="2026-09-23T19:20:24.514Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1572 command.oublock=32 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-23T19:20:34.149" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.471" grpc.time_ms=42.674 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.259613] gitaly[890]: time="2026-09-23T19:20:24.528Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:20:34.207" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.527" pid=890 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 191.260490] gitaly[890]: time="2026-09-23T19:20:24.528Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:20:34.207" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.527" grpc.time_ms=0.407 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.270992] gitaly[890]: time="2026-09-23T19:20:24.543Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:20:34.212" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.533" grpc.time_ms=10.545 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.334898] gitaly[890]: time="2026-09-23T19:20:24.607Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:20:34.287" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.607" pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.335660] gitaly[890]: time="2026-09-23T19:20:24.607Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:20:34.287" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.607" grpc.time_ms=0.231 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.394321] gitaly[890]: time="2026-09-23T19:20:24.667Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 diskcache=7f921c0c-70c5-4b28-9ff2-4003c7fad6f4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-23T19:20:34.306" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.627" pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.395222] gitaly[890]: time="2026-09-23T19:20:24.667Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1583 command.oublock=32 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-23T19:20:34.306" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.627" grpc.time_ms=40.053 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.420876] gitaly[890]: time="2026-09-23T19:20:24.693Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:20:34.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.693" pid=890 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 191.421637] gitaly[890]: time="2026-09-23T19:20:24.693Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:20:34.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.693" grpc.time_ms=0.346 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.451866] gitaly[890]: time="2026-09-23T19:20:24.724Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:20:34.025" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.715" grpc.time_ms=8.693 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.486523] gitaly[890]: time="2026-09-23T19:20:24.759Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:20:34.068" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.758" grpc.time_ms=0.271 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.491569] gitaly[890]: time="2026-09-23T19:20:24.764Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-23T19:20:34.073" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:24.763" grpc.time_ms=0.31 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 191.928264] gitaly[890]: time="2026-09-23T19:20:25.201Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCGZB1FEZNHWPE2HVVMA6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:20:35.004" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:25.195" grpc.time_ms=5.507 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 192.180498] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:25 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 1664 gitlab # [ 192.186429] setup-gitlab-start[2487]: {"id":1,"message":null,"error":null} gitlab # [ 193.072820] sidekiq-cluster[2470]: {"severity":"INFO","time":"2026-09-23T19:20:26.328Z","message":"Starting cluster with 1 processes"} gitlab # [ 197.525813] gitaly[890]: time="2026-09-23T19:20:30.798Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCQGNTQNHCGANPJCY1BMH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:20:40.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:30.787" grpc.time_ms=10.786 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.055616] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:31 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 841 gitlab # [ 198.059929] setup-gitlab-start[2199]: + mkdir -p /tmp/test-flake gitlab # [ 198.062503] setup-gitlab-start[2199]: + cd /tmp/test-flake gitlab # [ 198.063197] setup-gitlab-start[2199]: + git init -b master gitlab # [ 198.084697] setup-gitlab-start[2569]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 198.085352] setup-gitlab-start[2199]: + git config user.name test gitlab # [ 198.088536] setup-gitlab-start[2199]: + git config user.email test@example.com gitlab # [ 198.091345] setup-gitlab-start[2199]: + cat gitlab # [ 198.093464] setup-gitlab-start[2199]: + git add flake.nix gitlab # [ 198.096481] setup-gitlab-start[2199]: + git commit -m 'initial commit' gitlab # [ 198.117222] setup-gitlab-start[2574]: [master (root-commit) 0a68663] initial commit gitlab # [ 198.117920] setup-gitlab-start[2574]: 1 file changed, 10 insertions(+) gitlab # [ 198.118479] setup-gitlab-start[2574]: create mode 100644 flake.nix gitlab # [ 198.119821] setup-gitlab-start[2199]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 198.127701] setup-gitlab-start[2199]: + git push -u origin master gitlab # [ 198.205194] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:31 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 54 gitlab # [ 198.303254] gitaly[890]: time="2026-09-23T19:20:31.575Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRFSP5KQCC7WZHFMZVX8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:31.561" grpc.time_ms=13.89 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 198.316344] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:31 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 107 gitlab # [ 198.562347] gitaly[890]: time="2026-09-23T19:20:31.834Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:20:41.132" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:31.823" grpc.time_ms=11.117 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.606689] gitaly[890]: time="2026-09-23T19:20:31.879Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=329 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-23T19:21:01.066" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-23T19:20:31.867" grpc.time_ms=12.129 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 198.621913] gitaly[890]: time="2026-09-23T19:20:31.894Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M37VCRKKAV8H11BT0FYE2J2X duration_ms=132ns method=POST pid=890 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 198.673578] gitaly[890]: time="2026-09-23T19:20:31.946Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M37VCRKKAV8H11BT0FYE2J2X duration_ms=51ns method=POST pid=890 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 198.679838] gitaly[890]: time="2026-09-23T19:20:31.952Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:20:31.760" grpc.time_ms=191.597 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 198.737644] gitaly[890]: time="2026-09-23T19:20:32.010Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:20:32.009" grpc.time_ms=1.045 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 198.788519] gitaly[890]: time="2026-09-23T19:20:32.061Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:20:32.060" grpc.time_ms=0.572 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 198.823928] gitaly[890]: time="2026-09-23T19:20:32.096Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:20:32.095" grpc.time_ms=0.403 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 198.956336] gitaly[890]: time="2026-09-23T19:20:32.229Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:20:42.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:32.216" grpc.time_ms=11.888 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 199.015585] gitaly[890]: time="2026-09-23T19:20:32.288Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1562 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-23T19:20:42.057" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:20:32.267" grpc.time_ms=20.229 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 199.030465] gitaly[890]: time="2026-09-23T19:20:32.303Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M37VCRKKAV8H11BT0FYE2J2X duration_ms=177ns method=POST pid=890 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 199.031352] gitaly[890]: time="2026-09-23T19:20:32.304Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:20:32.125" grpc.time_ms=178.655 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 199.058208] gitaly[890]: time="2026-09-23T19:20:32.330Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X diskcache=8e448830-311b-4711-81c2-7741813fcf49 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:31.697" pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 199.067387] gitaly[890]: time="2026-09-23T19:20:32.331Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=212 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=892928 command.minflt=14032 command.oublock=32 command.real_time_ms=623 command.spawn_token_fork_ms=0 command.system_time_ms=136 command.user_time_ms=76 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:31.697" grpc.time_ms=633.567 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 199.079873] setup-gitlab-start[2578]: To http://gitlab/root/test-flake.git gitlab # [ 199.080151] setup-gitlab-start[2578]: * [new branch] master -> master gitlab # [ 199.080611] setup-gitlab-start[2578]: branch 'master' set up to track 'origin/master'. gitlab # [ 199.081415] gitlab-workhorse[825]: time="2026-09-23T19:20:32Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 199.081964] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:32 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 728 gitlab # [ 199.088141] systemd[1]: Finished setup-gitlab.service. gitlab # [ 199.090842] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 200.14 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 200.445162] nixbot[2658]: {"ts": "2026-09-23T19:20:33.717Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 200.481321] nixbot[2658]: {"ts": "2026-09-23T19:20:33.754Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 200.513132] nixbot[2658]: {"ts": "2026-09-23T19:20:33.785Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 200.518766] nixbot[2658]: {"ts": "2026-09-23T19:20:33.791Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 200.523792] nixbot[2658]: {"ts": "2026-09-23T19:20:33.795Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 200.530852] nixbot[2658]: {"ts": "2026-09-23T19:20:33.803Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 200.535998] nixbot[2658]: {"ts": "2026-09-23T19:20:33.807Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 200.542129] nixbot[2658]: {"ts": "2026-09-23T19:20:33.814Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 200.551485] nixbot[2658]: {"ts": "2026-09-23T19:20:33.822Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 200.553416] nixbot[2658]: {"ts": "2026-09-23T19:20:33.826Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 200.559837] nixbot[2658]: {"ts": "2026-09-23T19:20:33.832Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 200.569458] nixbot[2658]: {"ts": "2026-09-23T19:20:33.842Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 200.572119] nixbot[2658]: {"ts": "2026-09-23T19:20:33.844Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 200.574499] nixbot[2658]: {"ts": "2026-09-23T19:20:33.847Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 200.579157] nixbot[2658]: {"ts": "2026-09-23T19:20:33.851Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 200.583118] nixbot[2658]: {"ts": "2026-09-23T19:20:33.853Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 200.583549] nixbot[2658]: {"ts": "2026-09-23T19:20:33.855Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 200.584355] nixbot[2658]: {"ts": "2026-09-23T19:20:33.856Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 200.588701] nixbot[2658]: {"ts": "2026-09-23T19:20:33.860Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 200.589645] nixbot[2658]: {"ts": "2026-09-23T19:20:33.862Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 200.591161] nixbot[2658]: {"ts": "2026-09-23T19:20:33.863Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 200.594668] nixbot[2658]: {"ts": "2026-09-23T19:20:33.867Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 200.596478] nixbot[2658]: {"ts": "2026-09-23T19:20:33.869Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 200.598582] nixbot[2658]: {"ts": "2026-09-23T19:20:33.871Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 200.600208] nixbot[2658]: {"ts": "2026-09-23T19:20:33.873Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 200.603088] nixbot[2658]: {"ts": "2026-09-23T19:20:33.875Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 200.604398] nixbot[2658]: {"ts": "2026-09-23T19:20:33.877Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 200.605910] nixbot[2658]: {"ts": "2026-09-23T19:20:33.878Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 200.608173] nixbot[2658]: {"ts": "2026-09-23T19:20:33.880Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 200.622964] nixbot[2658]: {"ts": "2026-09-23T19:20:33.895Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 200.628648] nixbot[2658]: {"ts": "2026-09-23T19:20:33.901Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 200.632369] nixbot[2658]: {"ts": "2026-09-23T19:20:33.905Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 200.635980] nixbot[2658]: {"ts": "2026-09-23T19:20:33.908Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} gitlab # [ 200.639682] nixbot[2658]: {"ts": "2026-09-23T19:20:33.912Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"} gitlab # [ 200.644412] nixbot[2658]: {"ts": "2026-09-23T19:20:33.917Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"} gitlab # [ 200.652666] nixbot[2658]: {"ts": "2026-09-23T19:20:33.925Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"} gitlab # [ 200.657987] nixbot[2658]: {"ts": "2026-09-23T19:20:33.930Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"} gitlab # [ 200.665065] nixbot[2658]: {"ts": "2026-09-23T19:20:33.937Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "early_attribute_reporting"} gitlab # [ 200.675542] nixbot[2658]: {"ts": "2026-09-23T19:20:33.948Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 38, "migration": "terminal_report_reconciliation"} gitlab # [ 201.032638] nixbot[2658]: INFO: Started server process [2658] gitlab # [ 201.033718] nixbot[2658]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 201.052623] nixbot[2658]: INFO: ::ffff:127.0.0.1:51412 - "GET /health HTTP/1.1" 200 OK gitlab # [ 201.055514] systemd[1]: Started nixbot CI service. gitlab # [ 201.058960] systemd[1]: Reached target Multi-User System. gitlab # [ 201.059930] systemd[1]: Startup finished in 1.261s (kernel) + 3.599s (initrd) + 3min 16.198s (userspace) = 3min 21.059s. gitlab # [ 201.296951] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:34 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 244 gitlab # [ 201.303567] nixbot[2658]: {"ts": "2026-09-23T19:20:34.576Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 201.324678] nixbot[2658]: {"ts": "2026-09-23T19:20:34.597Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 201.415506] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 75 gitlab # [ 201.416274] nixbot[2658]: {"ts": "2026-09-23T19:20:34.688Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 201.417451] nixbot[2658]: {"ts": "2026-09-23T19:20:34.690Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 201.586802] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 156 gitlab # [ 201.593056] nixbot[2658]: {"ts": "2026-09-23T19:20:34.865Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab: (finished: waiting for unit nixbot.service, in 2.11 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 201.665199] nixbot[2658]: INFO: ::ffff:127.0.0.1:51418 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 201.719038] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:34 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 107 gitlab # [ 201.720646] nixbot[2658]: {"ts": "2026-09-23T19:20:34.993Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 201.743047] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 43 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.07 seconds) (finished: subtest: project discovered and webhook registered, in 0.08 seconds) subtest: push delivers the webhook gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' gitlab # [ 201.817351] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 83 gitlab # [ 201.822360] nixbot[2658]: {"ts": "2026-09-23T19:20:35.090Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 201.870931] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:35 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 48 gitlab # [ 201.875925] nixbot[2658]: {"ts": "2026-09-23T19:20:35.144Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 201.924673] gitaly[890]: time="2026-09-23T19:20:35.196Z" level=info msg="finished unary call with code OK" catfile.duration_ms=4 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=4 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=623 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCW2FH2ZCPYCCS80X4EZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-23T19:21:05.085" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:20:35.185" grpc.time_ms=10.829 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.954582] gitaly[890]: time="2026-09-23T19:20:35.207Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCW2FH2ZCPYCCS80X4EZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:05.106" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:20:35.206" grpc.time_ms=0.437 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.969766] gitaly[890]: time="2026-09-23T19:20:35.213Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCW2FH2ZCPYCCS80X4EZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-23T19:20:45.008" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:20:35.208" grpc.time_ms=4.75 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 201.981865] nixbot[2658]: {"ts": "2026-09-23T19:20:35.235Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 201.984338] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 82 gitlab # [ 202.021178] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 54 gitlab # [ 202.026240] nixbot[2658]: {"ts": "2026-09-23T19:20:35.298Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 202.035580] nixbot[2658]: {"ts": "2026-09-23T19:20:35.308Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 202.119611] gitaly[890]: time="2026-09-23T19:20:35.392Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VCW8J6A729PPSGRMDRMNS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:35.381" grpc.time_ms=10.723 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 202.121857] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:35 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 47 gitlab # [ 202.243985] gitaly[890]: time="2026-09-23T19:20:35.516Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCWB11TSAMYSAJ69CZF0V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:35.504" pid=890 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 202.244905] gitaly[890]: time="2026-09-23T19:20:35.516Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCWB11TSAMYSAJ69CZF0V grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:35.504" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 202.245920] gitlab-workhorse[825]: time="2026-09-23T19:20:35Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 202.248884] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:35 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 91 gitlab # [ 202.250421] gitaly[890]: time="2026-09-23T19:20:35.516Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=641 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCWB11TSAMYSAJ69CZF0V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:35.504" grpc.time_ms=11.895 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 202.382151] gitaly[890]: time="2026-09-23T19:20:35.654Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCWG3B1HT2HQJ4V9YDQ9C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:35.633" pid=890 remote_ip="2001:db8:1::1" request_sha=189e583caeef799b5b86d99e3e3346d2cfae3a1c response_bytes=223 span.kind=server system=grpc gitlab # [ 202.382404] gitaly[890]: time="2026-09-23T19:20:35.654Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1205 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCWG3B1HT2HQJ4V9YDQ9C grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:35.633" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 202.383597] gitaly[890]: time="2026-09-23T19:20:35.654Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1205 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCWG3B1HT2HQJ4V9YDQ9C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:35.633" grpc.time_ms=21.301 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 202.386324] gitlab-workhorse[825]: time="2026-09-23T19:20:35Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=223 gitlab # [ 202.388043] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:35 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 70 gitlab # [ 202.554192] gitaly[890]: time="2026-09-23T19:20:35.827Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M37VCWNE5QXBP2G210GK6JV1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:35.815" grpc.time_ms=11.263 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 202.556109] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:35 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 69 gitlab # [ 202.643234] gitaly[890]: time="2026-09-23T19:20:35.915Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCWRMZZ1YP3TVVBD1KR0X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:35.905" pid=890 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 202.643497] gitaly[890]: time="2026-09-23T19:20:35.916Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCWRMZZ1YP3TVVBD1KR0X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:35.905" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 202.644634] gitaly[890]: time="2026-09-23T19:20:35.916Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=646 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCWRMZZ1YP3TVVBD1KR0X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:35.905" grpc.time_ms=10.869 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 202.704870] gitlab-workhorse[825]: time="2026-09-23T19:20:35Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 202.705577] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:35 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 117 gitlab # [ 202.879918] gitaly[890]: time="2026-09-23T19:20:36.152Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=632 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VCWXJXJ3P6ESXE3YBNMKJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:36.140" grpc.time_ms=11.434 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 202.880876] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:36 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 134 gitlab # [ 202.956063] gitaly[890]: time="2026-09-23T19:20:36.228Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCX1WPPTPG3DDETNSR2BV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:36.213" pid=890 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 202.956368] gitaly[890]: time="2026-09-23T19:20:36.228Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCX1WPPTPG3DDETNSR2BV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:36.213" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.001453] gitaly[890]: time="2026-09-23T19:20:36.274Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCX1WPPTPG3DDETNSR2BV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:36.213" grpc.time_ms=60.111 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 203.002153] gitlab-workhorse[825]: time="2026-09-23T19:20:36Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 203.002941] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:36 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 118 gitlab # [ 203.098106] gitaly[890]: time="2026-09-23T19:20:36.370Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:06.269" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:20:36.369" grpc.time_ms=0.838 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.259839] gitaly[890]: time="2026-09-23T19:20:36.532Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-23T19:21:06.420" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:20:36.520" grpc.time_ms=11.654 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.290235] gitaly[890]: time="2026-09-23T19:20:36.563Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:20:46.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:20:36.552" grpc.time_ms=10.145 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.308945] gitaly[890]: time="2026-09-23T19:20:36.581Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=636 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:20:46.023" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:20:36.573" grpc.time_ms=7.775 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.390262] gitaly[890]: time="2026-09-23T19:20:36.663Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-23T19:20:46.111" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:36.662" grpc.time_ms=0.366 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.433394] gitaly[890]: time="2026-09-23T19:20:36.705Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=619 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-23T19:20:46.143" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:20:36.694" grpc.time_ms=11.435 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.501332] gitaly[890]: time="2026-09-23T19:20:36.774Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:20:36.773" grpc.time_ms=0.567 pid=890 span.kind=server system=grpc gitlab # [ 203.541785] gitaly[890]: time="2026-09-23T19:20:36.814Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:20:36.814" grpc.time_ms=0.442 pid=890 span.kind=server system=grpc gitlab # [ 203.564292] gitaly[890]: time="2026-09-23T19:20:36.837Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 diskcache=3f2ddb9a-5a63-40db-84a1-f5ba64babe79 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-23T19:20:46.157" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:36.707" pid=890 ref=refs/keep-around/0a686633e096d704cae69e46c6345dd0bcf8d3c3 remote_ip="2001:db8:1::1" revision=0a686633e096d704cae69e46c6345dd0bcf8d3c3 span.kind=server system=grpc username=root gitlab # [ 203.574973] gitaly[890]: time="2026-09-23T19:20:36.847Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=81 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5513 command.oublock=8 command.real_time_ms=119 command.spawn_token_fork_ms=21 command.system_time_ms=36 command.user_time_ms=45 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-23T19:20:46.157" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:20:36.707" grpc.time_ms=129.635 pid=890 ref=refs/keep-around/0a686633e096d704cae69e46c6345dd0bcf8d3c3 remote_ip="2001:db8:1::1" revision=0a686633e096d704cae69e46c6345dd0bcf8d3c3 span.kind=server system=grpc username=root gitlab # [ 203.593295] gitaly[890]: time="2026-09-23T19:20:36.865Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:06.765" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:20:36.865" grpc.time_ms=0.403 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 203.763580] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/0a686633e096d704cae69e46c6345dd0bcf8d3c3 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 709 gitlab # [ 203.764982] nixbot[2658]: {"ts": "2026-09-23T19:20:37.037Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/0a686633e096d704cae69e46c6345dd0bcf8d3c3 \"HTTP/1.1 201 Created\""} gitlab # [ 203.776157] nixbot[2658]: {"ts": "2026-09-23T19:20:37.048Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-0a686633e096-0"} gitlab # [ 203.931827] nixbot[2658]: {"ts": "2026-09-23T19:20:37.204Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-0a686633e096-0", "argv0": "bwrap"} gitlab # [ 204.016255] systemd[1]: Started Nix Daemon. gitlab # [ 204.091416] nix-daemon[2823]: accepted connection from pid 2818, user nixbot gitlab # [ 208.803346] nix-daemon[2823]: accepted connection from pid 2865, user nixbot gitlab # [ 208.823234] nix-daemon[2823]: accepted connection from pid 2818, user nixbot gitlab # [ 212.923934] nix-daemon[2823]: accepted connection from pid 2818, user nixbot gitlab # [ 213.014742] nix-daemon[2823]: accepted connection from pid 2908, user nixbot gitlab # [ 213.075692] gitaly[890]: time="2026-09-23T19:20:46.347Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VD6YD648M0B3HP3HJES1W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:16.246" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:20:46.346" grpc.time_ms=0.708 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.130603] gitaly[890]: time="2026-09-23T19:20:46.403Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VD6YD648M0B3HP3HJES1W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:20:56.180" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:20:46.390" grpc.time_ms=12.472 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.172463] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/0a686633e096d704cae69e46c6345dd0bcf8d3c3 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 157 gitlab # [ 213.173936] nixbot[2658]: {"ts": "2026-09-23T19:20:46.445Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/0a686633e096d704cae69e46c6345dd0bcf8d3c3 \"HTTP/1.1 201 Created\""} gitlab # [ 213.234932] gitaly[890]: time="2026-09-23T19:20:46.507Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VD73FTWCG86Y4TA72T1VK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:16.006" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:20:46.506" grpc.time_ms=0.647 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.316172] gitaly[890]: time="2026-09-23T19:20:46.588Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M37VD73FTWCG86Y4TA72T1VK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:20:56.258" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:20:46.578" grpc.time_ms=9.877 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 213.393537] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/0a686633e096d704cae69e46c6345dd0bcf8d3c3 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 218 gitlab # [ 213.399394] nixbot[2658]: {"ts": "2026-09-23T19:20:46.667Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/0a686633e096d704cae69e46c6345dd0bcf8d3c3 \"HTTP/1.1 201 Created\""} gitlab # [ 217.478850] nix-daemon[2823]: accepted connection from pid 2982, user nixbot gitlab # [ 217.558396] gitaly[890]: time="2026-09-23T19:20:50.830Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VDBB1KB3QSNW8F970W1K2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:20.028" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:20:50.829" grpc.time_ms=0.935 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.607638] gitaly[890]: time="2026-09-23T19:20:50.880Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M37VDBB1KB3QSNW8F970W1K2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:21:00.183" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:20:50.874" grpc.time_ms=5.787 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.668045] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:50 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/0a686633e096d704cae69e46c6345dd0bcf8d3c3 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 154 gitlab # [ 217.670862] nixbot[2658]: {"ts": "2026-09-23T19:20:50.942Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/0a686633e096d704cae69e46c6345dd0bcf8d3c3 \"HTTP/1.1 201 Created\""} gitlab # [ 217.717253] gitaly[890]: time="2026-09-23T19:20:50.989Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VDBG4GRQET17FCWXTWDKZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:20.188" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:20:50.988" grpc.time_ms=0.831 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.780397] gitaly[890]: time="2026-09-23T19:20:51.053Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M37VDBG4GRQET17FCWXTWDKZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:21:01.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:20:51.041" grpc.time_ms=11.337 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 217.868940] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:51 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/0a686633e096d704cae69e46c6345dd0bcf8d3c3 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 193 gitlab # [ 217.870162] nixbot[2658]: {"ts": "2026-09-23T19:20:51.142Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/0a686633e096d704cae69e46c6345dd0bcf8d3c3 \"HTTP/1.1 201 Created\""} gitlab # [ 217.922959] sidekiq-cluster[2530]: 2026-09-23T19:20:51.195Z pid=2530 tid=xu2 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2530, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>452493312} gitlab # [ 218.096945] sidekiq-cluster[2530]: W, [2026-09-23T19:20:51.369547 #2530] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 218.135392] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:20:51.408Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 218.240650] gitaly[890]: time="2026-09-23T19:20:51.513Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M37VDBZKCA9156E2BQRM4N6E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:51.500" grpc.time_ms=12.548 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.271058] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:51 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 100 gitlab # [ 218.334345] gitaly[890]: time="2026-09-23T19:20:51.607Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VDC2WZME1WXRC51YXTEK9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:51.599" pid=890 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 218.334608] gitaly[890]: time="2026-09-23T19:20:51.607Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VDC2WZME1WXRC51YXTEK9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:51.599" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.335666] gitaly[890]: time="2026-09-23T19:20:51.607Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=640 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VDC2WZME1WXRC51YXTEK9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:51.599" grpc.time_ms=8.419 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.336781] gitlab-workhorse[825]: time="2026-09-23T19:20:51Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 218.337857] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:51 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 60 gitlab # [ 218.506112] gitaly[890]: time="2026-09-23T19:20:51.778Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=634 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M37VDC7WGDF2X0KF8NEA40QN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:51.770" grpc.time_ms=8.605 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.506662] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:51 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 70 gitlab # [ 218.553451] gitaly[890]: time="2026-09-23T19:20:51.825Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VDCA5MCMAMBREDQ15NWJ5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:51.818" pid=890 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 218.553670] gitaly[890]: time="2026-09-23T19:20:51.826Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VDCA5MCMAMBREDQ15NWJ5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:51.818" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.554532] gitaly[890]: time="2026-09-23T19:20:51.826Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VDCA5MCMAMBREDQ15NWJ5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:20:51.818" grpc.time_ms=7.717 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 218.556188] gitlab-workhorse[825]: time="2026-09-23T19:20:51Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 218.557178] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:20:51 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 48 gitlab # [ 219.906341] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:20:53.178Z","message":"GitLab reliable fetch activated!"} gitlab # [ 219.906711] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:20:53.178Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 238.406822] postgres[3202]: [3202] ERROR: relation "table_without_models" does not exist gitlab # [ 238.407738] postgres[3202]: [3202] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 240.841449] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:14.110Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 240.841642] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:14.112Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [x86_64-linux]"} gitlab # [ 240.842368] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:14.113Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 240.843123] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:14.113Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 240.858046] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:14.130Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 39.51 seconds) gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 241.327163] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:14 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 23 gitlab # [ 241.393237] gitaly[890]: time="2026-09-23T19:21:14.665Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M37VE2KDZSSF8E65P2YQ1HM5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:14.651" grpc.time_ms=13.725 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 241.394709] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:14 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 60 gitlab # Enumerating objects: 5, done. gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 273.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 241.704460] gitaly[890]: time="2026-09-23T19:21:14.977Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=326 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-23T19:21:44.864" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-23T19:21:14.965" grpc.time_ms=11.479 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 241.741395] gitaly[890]: time="2026-09-23T19:21:15.014Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ duration_ms=134ns method=POST pid=890 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 241.765405] gitaly[890]: time="2026-09-23T19:21:15.038Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ duration_ms=23ns method=POST pid=890 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 241.765830] gitaly[890]: time="2026-09-23T19:21:15.038Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5312 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:14.878" grpc.time_ms=159.819 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 241.827953] gitaly[890]: time="2026-09-23T19:21:15.099Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:15.099" grpc.time_ms=0.429 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 241.865742] gitaly[890]: time="2026-09-23T19:21:15.138Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4781 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:15.137" grpc.time_ms=1.078 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 241.895770] gitaly[890]: time="2026-09-23T19:21:15.168Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:15.168" grpc.time_ms=0.387 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 241.956625] gitaly[890]: time="2026-09-23T19:21:15.229Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ duration_ms=33ns method=POST pid=890 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 241.959619] gitaly[890]: time="2026-09-23T19:21:15.232Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4806 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:15.195" grpc.time_ms=36.739 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 241.983385] gitaly[890]: time="2026-09-23T19:21:15.256Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ diskcache=078999bd-3e49-4a1e-956d-93dd1309611a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:14.799" pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 241.984251] gitaly[890]: time="2026-09-23T19:21:15.256Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=219 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=868352 command.minflt=13471 command.oublock=32 command.real_time_ms=446 command.spawn_token_fork_ms=0 command.system_time_ms=155 command.user_time_ms=64 component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:14.799" grpc.time_ms=456.407 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 242.004685] gitlab-workhorse[825]: time="2026-09-23T19:21:15Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 242.005233] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:15 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 554 gitlab # To http://gitlab/root/test-flake.git gitlab # 0a68663..263054a master -> master gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.76 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1' gitlab # [ 242.080376] nixbot[2658]: INFO: ::ffff:127.0.0.1:37440 - "GET /api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1', in 0.06 seconds) gitlab # [ 242.151804] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:15.423Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e494039080e7b576c1edd773","created_at":"2026-09-23T19:20:24.553Z","trace_propagation_headers":{"sentry-trace":"b486593853604b4f91db8f9fca63c5d2-4e9d4d469baa4c93","baggage":"sentry-trace_id=b486593853604b4f91db8f9fca63c5d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCGZB1FEZNHWPE2HVVMA6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:24.580Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e494039080e7b576c1edd773: start","job_status":"start","queue_duration_s":50.843078,"scheduling_latency_s":50.843118} gitlab # [ 242.378950] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:15.651Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"e494039080e7b576c1edd773","created_at":"2026-09-23T19:20:24.553Z","trace_propagation_headers":{"sentry-trace":"b486593853604b4f91db8f9fca63c5d2-4e9d4d469baa4c93","baggage":"sentry-trace_id=b486593853604b4f91db8f9fca63c5d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCGZB1FEZNHWPE2HVVMA6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:24.580Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-e494039080e7b576c1edd773: done: 0.227892 sec","job_status":"done","queue_duration_s":50.843078,"scheduling_latency_s":50.843118,"redis_calls":25,"redis_duration_s":0.008287,"redis_read_bytes":912,"redis_write_bytes":2805,"redis_feature_flag_calls":15,"redis_feature_flag_duration_s":0.00488,"redis_feature_flag_read_bytes":904,"redis_feature_flag_write_bytes":1640,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001973,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001434,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":11,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.089946348,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.227892,"completed_at":"2026-09-23T19:21:15.651Z","load_balancing_strategy":"primary","db_duration_s":0.062032,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 242.380463] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:15.653Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"84e3b5536bba3fa4e928b0c5","created_at":"2026-09-23T19:20:24.595Z","trace_propagation_headers":{"sentry-trace":"b486593853604b4f91db8f9fca63c5d2-4e9d4d469baa4c93","baggage":"sentry-trace_id=b486593853604b4f91db8f9fca63c5d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCGZB1FEZNHWPE2HVVMA6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:24.602Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-84e3b5536bba3fa4e928b0c5: start","job_status":"start","queue_duration_s":51.05081,"scheduling_latency_s":51.050827} gitlab # [ 242.560634] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:15.833Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"84e3b5536bba3fa4e928b0c5","created_at":"2026-09-23T19:20:24.595Z","trace_propagation_headers":{"sentry-trace":"b486593853604b4f91db8f9fca63c5d2-4e9d4d469baa4c93","baggage":"sentry-trace_id=b486593853604b4f91db8f9fca63c5d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCGZB1FEZNHWPE2HVVMA6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:24.602Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-84e3b5536bba3fa4e928b0c5: done: 0.180157 sec","job_status":"done","queue_duration_s":51.05081,"scheduling_latency_s":51.050827,"redis_calls":14,"redis_duration_s":0.079568,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.077084,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001414,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.071724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.082292303,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.180157,"completed_at":"2026-09-23T19:21:15.833Z","load_balancing_strategy":"primary","db_duration_s":0.026073,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 242.632081] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:15.904Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ec7b181ea21c54d368bc8eab","created_at":"2026-09-23T19:20:25.130Z","trace_propagation_headers":{"sentry-trace":"b486593853604b4f91db8f9fca63c5d2-4e9d4d469baa4c93","baggage":"sentry-trace_id=b486593853604b4f91db8f9fca63c5d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCGZB1FEZNHWPE2HVVMA6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:25.134Z","job_size_bytes":5,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ec7b181ea21c54d368bc8eab: start","job_status":"start","queue_duration_s":50.769802,"scheduling_latency_s":50.769833} gitlab # [ 242.790865] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:16.063Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"ec7b181ea21c54d368bc8eab","created_at":"2026-09-23T19:20:25.130Z","trace_propagation_headers":{"sentry-trace":"b486593853604b4f91db8f9fca63c5d2-4e9d4d469baa4c93","baggage":"sentry-trace_id=b486593853604b4f91db8f9fca63c5d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCGZB1FEZNHWPE2HVVMA6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:25.134Z","job_size_bytes":5,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-ec7b181ea21c54d368bc8eab: done: 0.159114 sec","job_status":"done","queue_duration_s":50.769802,"scheduling_latency_s":50.769833,"redis_calls":12,"redis_duration_s":0.003575,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001136,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00188,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000559,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.108655,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000277689,"duration_s":0.159114,"completed_at":"2026-09-23T19:21:16.063Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027265,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 242.792755] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:16.064Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"f863ae9aad358c77d93d47f1","created_at":"2026-09-23T19:20:25.151Z","trace_propagation_headers":{"sentry-trace":"b486593853604b4f91db8f9fca63c5d2-4e9d4d469baa4c93","baggage":"sentry-trace_id=b486593853604b4f91db8f9fca63c5d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCGZB1FEZNHWPE2HVVMA6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:25.154Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-f863ae9aad358c77d93d47f1: start","job_status":"start","queue_duration_s":50.910658,"scheduling_latency_s":50.910673} gitlab # [ 242.940621] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:16.212Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"f863ae9aad358c77d93d47f1","created_at":"2026-09-23T19:20:25.151Z","trace_propagation_headers":{"sentry-trace":"b486593853604b4f91db8f9fca63c5d2-4e9d4d469baa4c93","baggage":"sentry-trace_id=b486593853604b4f91db8f9fca63c5d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCGZB1FEZNHWPE2HVVMA6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:25.154Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-f863ae9aad358c77d93d47f1: done: 0.148167 sec","job_status":"done","queue_duration_s":50.910658,"scheduling_latency_s":50.910673,"redis_calls":11,"redis_duration_s":0.002446,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000642,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001377,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.123,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.123,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.078354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000291377,"duration_s":0.148167,"completed_at":"2026-09-23T19:21:16.212Z","load_balancing_strategy":"primary","db_duration_s":0.069619,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.942388] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:16.214Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"3bcf095010122cf926ae4e01","created_at":"2026-09-23T19:20:25.156Z","trace_propagation_headers":{"sentry-trace":"b486593853604b4f91db8f9fca63c5d2-4e9d4d469baa4c93","baggage":"sentry-trace_id=b486593853604b4f91db8f9fca63c5d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCGZB1FEZNHWPE2HVVMA6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:25.158Z","job_size_bytes":89,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-3bcf095010122cf926ae4e01: start","job_status":"start","queue_duration_s":51.05556,"scheduling_latency_s":51.055575} gitlab # [ 243.017465] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:16.239Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"3bcf095010122cf926ae4e01","created_at":"2026-09-23T19:20:25.156Z","trace_propagation_headers":{"sentry-trace":"b486593853604b4f91db8f9fca63c5d2-4e9d4d469baa4c93","baggage":"sentry-trace_id=b486593853604b4f91db8f9fca63c5d2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCGZB1FEZNHWPE2HVVMA6","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:25.158Z","job_size_bytes":89,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-3bcf095010122cf926ae4e01: done: 0.025339 sec","job_status":"done","queue_duration_s":51.05556,"scheduling_latency_s":51.055575,"redis_calls":11,"redis_duration_s":0.002219,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000632,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001181,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000406,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000287187,"duration_s":0.025339,"completed_at":"2026-09-23T19:21:16.239Z","load_balancing_strategy":"primary","db_duration_s":0.005059,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.019878] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:16.241Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 0a686633e096d704cae69e46c6345dd0bcf8d3c3 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a82867f5aa5f8dc67b8bfada","created_at":"2026-09-23T19:20:32.167Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-73ef935c16004bfe","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:32.188Z","job_size_bytes":185,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a82867f5aa5f8dc67b8bfada: start","job_status":"start","queue_duration_s":44.052419,"scheduling_latency_s":44.052431} gitlab # [ 243.035901] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:16.251Z","message":"changes: \"0000000000000000000000000000000000000000 0a686633e096d704cae69e46c6345dd0bcf8d3c3 refs/heads/master\\n\""} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1' gitlab # [ 243.085473] gitaly[890]: time="2026-09-23T19:21:16.357Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:21:26.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:16.350" grpc.time_ms=6.392 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.112064] gitaly[890]: time="2026-09-23T19:21:16.384Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=628 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:21:26.035" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:21:16.376" grpc.time_ms=8.585 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.125067] nixbot[2658]: INFO: ::ffff:127.0.0.1:52646 - "GET /api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1', in 0.04 seconds) gitlab # [ 243.183697] gitaly[890]: time="2026-09-23T19:21:16.456Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=645 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-23T19:21:26.107" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:16.448" grpc.time_ms=8.23 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.239234] gitaly[890]: time="2026-09-23T19:21:16.511Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3933 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:16.511" grpc.time_ms=0.633 pid=890 span.kind=server system=grpc gitlab # [ 243.278124] gitaly[890]: time="2026-09-23T19:21:16.550Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:16.550" grpc.time_ms=0.364 pid=890 span.kind=server system=grpc gitlab # [ 243.297394] gitaly[890]: time="2026-09-23T19:21:16.570Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X diskcache=bdb3b8b1-77aa-466d-932f-3d3be195e440 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-23T19:21:26.139" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:21:16.479" pid=890 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 243.297995] gitaly[890]: time="2026-09-23T19:21:16.570Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=76 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5149 command.oublock=8 command.real_time_ms=81 command.spawn_token_fork_ms=0 command.system_time_ms=46 command.user_time_ms=30 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-23T19:21:26.139" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:21:16.479" grpc.time_ms=90.711 pid=890 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 243.318587] gitaly[890]: time="2026-09-23T19:21:16.591Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-23T19:21:26.250" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:21:16.590" grpc.time_ms=0.256 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.343969] gitaly[890]: time="2026-09-23T19:21:16.616Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=940 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-23T19:21:26.261" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:16.601" grpc.time_ms=14.852 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.435080] gitaly[890]: time="2026-09-23T19:21:16.707Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=338 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-23T19:21:46.098" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:16.698" grpc.time_ms=8.7 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.483532] gitaly[890]: time="2026-09-23T19:21:16.756Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-23T19:21:26.004" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:21:16.755" grpc.time_ms=0.46 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.515913] gitaly[890]: time="2026-09-23T19:21:16.788Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=340 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-23T19:21:26.029" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-23T19:21:16.780" grpc.time_ms=7.957 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.542449] gitaly[890]: time="2026-09-23T19:21:16.815Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:21:26.054" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:16.805" grpc.time_ms=9.743 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.562258] gitaly[890]: time="2026-09-23T19:21:16.835Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:21:26.073" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:16.823" grpc.time_ms=11.318 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.594883] gitaly[890]: time="2026-09-23T19:21:16.867Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=960 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-23T19:21:46.250" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:16.850" grpc.time_ms=16.639 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 243.966740] gitaly[890]: time="2026-09-23T19:21:17.239Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-23T19:21:27.007" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:17.238" grpc.time_ms=1.27 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 244.018625] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:17.289Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 0a686633e096d704cae69e46c6345dd0bcf8d3c3 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"a82867f5aa5f8dc67b8bfada","created_at":"2026-09-23T19:20:32.167Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-73ef935c16004bfe","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:20:32.188Z","job_size_bytes":185,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-a82867f5aa5f8dc67b8bfada: done: 1.048865 sec","job_status":"done","queue_duration_s":44.052419,"scheduling_latency_s":44.052431,"gitaly_calls":13,"gitaly_duration_s":0.371256,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.055077,"redis_read_bytes":2103,"redis_write_bytes":21750,"redis_cache_calls":15,"redis_cache_duration_s":0.004535,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.006301,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001656,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7838,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002896,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.02795,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.011739,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.056,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.187,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.208,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.056,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.554879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00043581,"exclusive_lock_hold_duration_s":0.112962961,"duration_s":1.048865,"completed_at":"2026-09-23T19:21:17.289Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.096783,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 244.065945] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:17.338Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"dd23570411a8449d07b4a7e7","created_at":"2026-09-23T19:20:36.877Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-6e5fe66cbcdd42b3","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:36.887Z","job_size_bytes":65,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-dd23570411a8449d07b4a7e7: start","job_status":"start","queue_duration_s":40.451002,"scheduling_latency_s":40.451048} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1' gitlab # [ 244.166408] nixbot[2658]: INFO: ::ffff:127.0.0.1:52648 - "GET /api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1', in 0.04 seconds) gitlab # [ 244.189847] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:17.462Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"dd23570411a8449d07b4a7e7","created_at":"2026-09-23T19:20:36.877Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-6e5fe66cbcdd42b3","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:36.887Z","job_size_bytes":65,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-dd23570411a8449d07b4a7e7: done: 0.123888 sec","job_status":"done","queue_duration_s":40.451002,"scheduling_latency_s":40.451048,"redis_calls":16,"redis_duration_s":0.004956,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002771,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001678,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000507,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000548393,"duration_s":0.123888,"completed_at":"2026-09-23T19:21:17.461Z","load_balancing_strategy":"primary","db_duration_s":0.023485,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 244.258823] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:17.530Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"93d5004d8d997327928b5614","created_at":"2026-09-23T19:20:36.888Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-6e5fe66cbcdd42b3","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:36.894Z","job_size_bytes":65,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-93d5004d8d997327928b5614: start","job_status":"start","queue_duration_s":40.63651,"scheduling_latency_s":40.636561} gitlab # [ 244.374702] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:17.647Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"93d5004d8d997327928b5614","created_at":"2026-09-23T19:20:36.888Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-6e5fe66cbcdd42b3","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:36.894Z","job_size_bytes":65,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-93d5004d8d997327928b5614: done: 0.116519 sec","job_status":"done","queue_duration_s":40.63651,"scheduling_latency_s":40.636561,"redis_calls":21,"redis_duration_s":0.006432,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.003001,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001562,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001869,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.070206,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000282718,"duration_s":0.116519,"completed_at":"2026-09-23T19:21:17.647Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033097,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.448802] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:17.721Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5da4362e84d51aa0ff66c670","created_at":"2026-09-23T19:20:37.018Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-6e5fe66cbcdd42b3","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:37.023Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5da4362e84d51aa0ff66c670: start","job_status":"start","queue_duration_s":40.697926,"scheduling_latency_s":40.697957} gitlab # [ 244.634147] gitaly[890]: time="2026-09-23T19:21:17.906Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:17.904" grpc.time_ms=2.171 pid=890 span.kind=server system=grpc gitlab # [ 244.674259] gitaly[890]: time="2026-09-23T19:21:17.947Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:17.946" grpc.time_ms=0.295 pid=890 span.kind=server system=grpc gitlab # [ 244.715420] gitaly[890]: time="2026-09-23T19:21:17.988Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:17.987" grpc.time_ms=0.298 pid=890 span.kind=server system=grpc gitlab # [ 244.736636] gitaly[890]: time="2026-09-23T19:21:18.009Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 diskcache=d50b9165-a4af-4074-97b7-d010d1e4578a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:21:47.269" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:17.869" pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 244.737229] gitaly[890]: time="2026-09-23T19:21:18.009Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=157 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7515 command.oublock=0 command.real_time_ms=129 command.spawn_token_fork_ms=0 command.system_time_ms=120 command.user_time_ms=37 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:21:47.269" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:17.869" grpc.time_ms=139.925 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 244.892151] gitaly[890]: time="2026-09-23T19:21:18.164Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-23T19:21:48.063" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:18.163" grpc.time_ms=0.817 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 244.931288] gitaly[890]: time="2026-09-23T19:21:18.204Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCX7669597E0KHBJ945B0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:48.102" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:18.202" grpc.time_ms=0.898 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 244.945128] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:18.217Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5da4362e84d51aa0ff66c670","created_at":"2026-09-23T19:20:37.018Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-6e5fe66cbcdd42b3","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:37.023Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2530,"job_status":"rescheduled","message":"PipelineProcessWorker JID-5da4362e84d51aa0ff66c670: rescheduled"} gitlab # [ 244.956719] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:18.229Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"5da4362e84d51aa0ff66c670","created_at":"2026-09-23T19:20:37.018Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-6e5fe66cbcdd42b3","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-23T19:20:37.023Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5da4362e84d51aa0ff66c670: done: 0.507711 sec","job_status":"done","queue_duration_s":40.697926,"scheduling_latency_s":40.697957,"gitaly_calls":3,"gitaly_duration_s":0.149562,"redis_calls":120,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.0281,"redis_read_bytes":460,"redis_write_bytes":29295,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001496,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001114,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.010361,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":27,"redis_queues_duration_s":0.004348,"redis_queues_read_bytes":37,"redis_queues_write_bytes":15582,"redis_queues_metadata_calls":22,"redis_queues_metadata_duration_s":0.005975,"redis_queues_metadata_read_bytes":34,"redis_queues_metadata_write_bytes":3577,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000561,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000496,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002253,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.018,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.025,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.031,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.248636,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.417620396,"duration_s":0.507711,"completed_at":"2026-09-23T19:21:18.228Z","load_balancing_strategy":"primary","db_duration_s":0.075276,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 245.007800] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:18.231Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0a686633e096d704cae69e46c6345dd0bcf8d3c3 263054a42f00ab5c2554e36527725bb48ab5e4c1 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"15e31bc58b42fffefa576335","created_at":"2026-09-23T19:21:15.216Z","trace_propagation_headers":{"sentry-trace":"ceb0b6f3e456474babb7761654ca1b6a-a2c51388a6af45de","baggage":"sentry-trace_id=ceb0b6f3e456474babb7761654ca1b6a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VE2Q2PTE6S3JA2024M7JJ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:15.218Z","job_size_bytes":185,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-15e31bc58b42fffefa576335: start","job_status":"start","queue_duration_s":3.012774,"scheduling_latency_s":3.012806} gitlab # [ 245.008656] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:18.242Z","message":"changes: \"0a686633e096d704cae69e46c6345dd0bcf8d3c3 263054a42f00ab5c2554e36527725bb48ab5e4c1 refs/heads/master\\n\""} gitlab # [ 245.009140] gitaly[890]: time="2026-09-23T19:21:18.264Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:21:28.024" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:18.255" grpc.time_ms=9.399 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.070187] gitaly[890]: time="2026-09-23T19:21:18.342Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-23T19:21:28.101" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:21:18.332" grpc.time_ms=10.694 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.074238] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:18.346Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"031e1a9fdc60c943b01a69b0","created_at":"2026-09-23T19:21:18.344Z","trace_propagation_headers":{"sentry-trace":"ceb0b6f3e456474babb7761654ca1b6a-dc1a53d4b9f449e0","baggage":"sentry-trace_id=ceb0b6f3e456474babb7761654ca1b6a,sentry-release=Unknown"},"correlation_id":"01M37VE2Q2PTE6S3JA2024M7JJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"6939512aba4e69af5098d401","job_size_bytes":46,"pid":2530,"job_status":"deduplicated","message":"ProjectCacheWorker JID-031e1a9fdc60c943b01a69b0: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 245.091157] gitaly[890]: time="2026-09-23T19:21:18.363Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=637 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-23T19:21:28.014" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:18.355" grpc.time_ms=8.599 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.117177] gitaly[890]: time="2026-09-23T19:21:18.389Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=653 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-23T19:21:48.282" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:18.382" grpc.time_ms=6.978 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.131103] gitaly[890]: time="2026-09-23T19:21:18.403Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=348 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-23T19:21:28.059" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-23T19:21:18.399" grpc.time_ms=3.689 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1' gitlab # [ 245.163268] gitaly[890]: time="2026-09-23T19:21:18.435Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:21:28.084" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:18.424" grpc.time_ms=10.944 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.204739] nixbot[2658]: INFO: ::ffff:127.0.0.1:52654 - "GET /api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1', in 0.04 seconds) gitlab # [ 245.272484] gitaly[890]: time="2026-09-23T19:21:18.545Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-23T19:21:28.202" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:18.543" grpc.time_ms=1.758 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.300961] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:18.568Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0a686633e096d704cae69e46c6345dd0bcf8d3c3 263054a42f00ab5c2554e36527725bb48ab5e4c1 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"15e31bc58b42fffefa576335","created_at":"2026-09-23T19:21:15.216Z","trace_propagation_headers":{"sentry-trace":"ceb0b6f3e456474babb7761654ca1b6a-a2c51388a6af45de","baggage":"sentry-trace_id=ceb0b6f3e456474babb7761654ca1b6a,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VE2Q2PTE6S3JA2024M7JJ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:15.218Z","job_size_bytes":185,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-15e31bc58b42fffefa576335: done: 0.337636 sec","job_status":"done","queue_duration_s":3.012774,"scheduling_latency_s":3.012806,"gitaly_calls":7,"gitaly_duration_s":0.147698,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.025244000000000003,"redis_read_bytes":501,"redis_write_bytes":11941,"redis_cache_calls":9,"redis_cache_duration_s":0.003242,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001084,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5718,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004149,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.007551,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.009218,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.031,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.031,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.137595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000461511,"exclusive_lock_hold_duration_s":0.079996759,"duration_s":0.337636,"completed_at":"2026-09-23T19:21:18.568Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026385,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 245.309890] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:18.570Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"6939512aba4e69af5098d401","created_at":"2026-09-23T19:21:16.399Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-6ca7e033c0384101","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:16.421Z","job_size_bytes":46,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-6939512aba4e69af5098d401: start","job_status":"start","queue_duration_s":2.149373,"scheduling_latency_s":2.149403} gitlab # [ 245.425325] gitaly[890]: time="2026-09-23T19:21:18.697Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-24T01:21:18.691" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:21:18.691" grpc.time_ms=5.591 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.467719] gitaly[890]: time="2026-09-23T19:21:18.740Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=655 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-23T19:21:48.127" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:18.728" grpc.time_ms=11.843 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.527630] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:18.799Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"6939512aba4e69af5098d401","created_at":"2026-09-23T19:21:16.399Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-6ca7e033c0384101","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:16.421Z","job_size_bytes":46,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-6939512aba4e69af5098d401: done: 0.22902 sec","job_status":"done","queue_duration_s":2.149373,"scheduling_latency_s":2.149403,"gitaly_calls":2,"gitaly_duration_s":0.048588,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.09104799999999999,"redis_read_bytes":69,"redis_write_bytes":4885,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.065119,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000651,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2113,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001368,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.023082,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000828,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000453409,"duration_s":0.22902,"completed_at":"2026-09-23T19:21:18.799Z","load_balancing_strategy":"primary","db_duration_s":0.013844,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 245.598493] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:18.870Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"4154acce85a438280f8d0a5e","created_at":"2026-09-23T19:21:16.438Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-6ca7e033c0384101","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:16.443Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-4154acce85a438280f8d0a5e: start","job_status":"start","queue_duration_s":2.427479,"scheduling_latency_s":2.427527} gitlab # [ 245.677767] gitaly[890]: time="2026-09-23T19:21:18.949Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X diskcache=6a6fe62e-1b84-40c6-89a7-280641b9630c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-24T01:21:18.914" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:18.914" pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.678482] gitaly[890]: time="2026-09-23T19:21:18.950Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1283 command.oublock=0 command.real_time_ms=35 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-24T01:21:18.914" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:18.914" grpc.time_ms=35.227 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.766096] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:19.038Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"4154acce85a438280f8d0a5e","created_at":"2026-09-23T19:21:16.438Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-6ca7e033c0384101","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:16.443Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-4154acce85a438280f8d0a5e: done: 0.167499 sec","job_status":"done","queue_duration_s":2.427479,"scheduling_latency_s":2.427527,"gitaly_calls":1,"gitaly_duration_s":0.075372,"redis_calls":15,"redis_duration_s":0.004049,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000416,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001664,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000605,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001364,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.013,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.025,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.122103229,"duration_s":0.167499,"completed_at":"2026-09-23T19:21:19.038Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018284,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 245.835801] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:19.107Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"95d0155e52b412a2284ae324","created_at":"2026-09-23T19:21:16.585Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-6ca7e033c0384101","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:16.587Z","job_size_bytes":23,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-95d0155e52b412a2284ae324: start","job_status":"start","queue_duration_s":2.519916,"scheduling_latency_s":2.519947} gitlab # [ 245.888195] gitaly[890]: time="2026-09-23T19:21:19.160Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=649 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-23T19:21:49.048" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:19.148" grpc.time_ms=11.301 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.913238] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:19.185Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"95d0155e52b412a2284ae324","created_at":"2026-09-23T19:21:16.585Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-6ca7e033c0384101","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:16.587Z","job_size_bytes":23,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-95d0155e52b412a2284ae324: done: 0.077631 sec","job_status":"done","queue_duration_s":2.519916,"scheduling_latency_s":2.519947,"gitaly_calls":1,"gitaly_duration_s":0.014674,"redis_calls":16,"redis_duration_s":0.016773,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.00042,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001823,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.013356,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.077631,"completed_at":"2026-09-23T19:21:19.185Z","load_balancing_strategy":"primary","db_duration_s":0.004052,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 245.915441] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:19.187Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"45d037f5540b97c770a8746c","created_at":"2026-09-23T19:21:17.103Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-6ca7e033c0384101","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d3a15de2a2a5bd5bdb5d28239931be1c9ed1ad1dd6e979d0b8fabf140d052d08","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:17.110Z","job_size_bytes":467,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-45d037f5540b97c770a8746c: start","job_status":"start","queue_duration_s":2.077495,"scheduling_latency_s":2.077534} gitlab # [ 245.966800] gitaly[890]: time="2026-09-23T19:21:19.227Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:49.125" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:19.226" grpc.time_ms=0.855 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 245.969325] gitaly[890]: time="2026-09-23T19:21:19.242Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-23T19:21:29.009" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:19.230" grpc.time_ms=11.529 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.062438] gitaly[890]: time="2026-09-23T19:21:19.335Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-23T19:21:49.233" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:19.333" grpc.time_ms=1.159 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.068259] gitaly[890]: time="2026-09-23T19:21:19.341Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=23 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-23T19:21:49.237" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:19.337" grpc.time_ms=3.067 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.170118] gitaly[890]: time="2026-09-23T19:21:19.442Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:21:29.212" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:19.432" grpc.time_ms=9.932 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1' gitlab # [ 246.252735] nixbot[2658]: INFO: ::ffff:127.0.0.1:52670 - "GET /api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1', in 0.05 seconds) gitlab # [ 246.645939] gitaly[890]: time="2026-09-23T19:21:19.918Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=967 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-23T19:21:49.801" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:19.901" grpc.time_ms=16.954 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.662248] gitaly[890]: time="2026-09-23T19:21:19.934Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=633 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VCRKKAV8H11BT0FYE2J2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-23T19:21:49.820" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:19.920" grpc.time_ms=13.943 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 246.681241] sidekiq-cluster[2530]: {"severity":"WARN","time":"2026-09-23T19:21:19.953Z","class":"CreatePipelineWorker","correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"0a686633e096d704cae69e46c6345dd0bcf8d3c3","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"0a686633e096d704cae69e46c6345dd0bcf8d3c3","ref":"refs/heads/master"},"retry":0} gitlab # [ 246.685223] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:19.957Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"45d037f5540b97c770a8746c","created_at":"2026-09-23T19:21:17.103Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-6ca7e033c0384101","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d3a15de2a2a5bd5bdb5d28239931be1c9ed1ad1dd6e979d0b8fabf140d052d08","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:17.110Z","job_size_bytes":467,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-45d037f5540b97c770a8746c: done: 0.770018 sec","job_status":"done","queue_duration_s":2.077495,"scheduling_latency_s":2.077534,"gitaly_calls":7,"gitaly_duration_s":0.101837,"redis_calls":51,"redis_duration_s":0.01506,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.001217,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004343,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001287,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.007288,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.624466,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000233829,"duration_s":0.770018,"completed_at":"2026-09-23T19:21:19.957Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.051473,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 246.734857] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:19.959Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"3058b04e-ec2e-441e-91e3-9d0c31e6a15f\"}"],"class":"WebHookWorker","jid":"a63b4a459b7b76a1e9d651db","created_at":"2026-09-23T19:21:17.214Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-6ca7e033c0384101","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6e434776922305263061ef3b99fc1e0ac0ea41e7e5a3c93a66c0979ceb828f1c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:17.219Z","job_size_bytes":1751,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-a63b4a459b7b76a1e9d651db: start","job_status":"start","queue_duration_s":2.739607,"scheduling_latency_s":2.739623} gitlab # [ 246.794746] nixbot[2658]: INFO: ::ffff:127.0.0.1:52686 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 246.807289] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.079Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"3058b04e-ec2e-441e-91e3-9d0c31e6a15f\"}"],"class":"WebHookWorker","jid":"a63b4a459b7b76a1e9d651db","created_at":"2026-09-23T19:21:17.214Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-6ca7e033c0384101","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6e434776922305263061ef3b99fc1e0ac0ea41e7e5a3c93a66c0979ceb828f1c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:17.219Z","job_size_bytes":1751,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-a63b4a459b7b76a1e9d651db: done: 0.120741 sec","job_status":"done","queue_duration_s":2.739607,"scheduling_latency_s":2.739623,"redis_calls":19,"redis_duration_s":0.003285,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000787,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000398,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001433,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000667,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.05467538499999591,"cpu_s":0.055809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000262603,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.120741,"completed_at":"2026-09-23T19:21:20.079Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002446,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 246.812284] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.084Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8303d39ca00023c10433def4","created_at":"2026-09-23T19:21:17.858Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:17.861Z","job_size_bytes":23,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8303d39ca00023c10433def4: start","job_status":"start","queue_duration_s":2.223257,"scheduling_latency_s":2.223277} gitlab # [ 246.866171] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.138Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8303d39ca00023c10433def4","created_at":"2026-09-23T19:21:17.858Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:17.861Z","job_size_bytes":23,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8303d39ca00023c10433def4: done: 0.053805 sec","job_status":"done","queue_duration_s":2.223257,"scheduling_latency_s":2.223277,"redis_calls":10,"redis_duration_s":0.002677,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000357,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001824,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000496,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000238299,"duration_s":0.053805,"completed_at":"2026-09-23T19:21:20.138Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013007,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 246.867416] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.140Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"ac0a8a7b710dffa6d4e7cc42","created_at":"2026-09-23T19:21:18.014Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.018Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-ac0a8a7b710dffa6d4e7cc42: start","job_status":"start","queue_duration_s":2.1219,"scheduling_latency_s":2.121915} gitlab # [ 246.891915] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.164Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"ac0a8a7b710dffa6d4e7cc42","created_at":"2026-09-23T19:21:18.014Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.018Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-ac0a8a7b710dffa6d4e7cc42: done: 0.024501 sec","job_status":"done","queue_duration_s":2.1219,"scheduling_latency_s":2.121915,"redis_calls":10,"redis_duration_s":0.002184,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000409,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001343,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000432,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012748,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000293054,"duration_s":0.024501,"completed_at":"2026-09-23T19:21:20.164Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006895,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 246.940198] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.165Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"223d05f9e372392d119b1cf4","created_at":"2026-09-23T19:21:18.026Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.035Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-223d05f9e372392d119b1cf4: start","job_status":"start","queue_duration_s":2.130294,"scheduling_latency_s":2.130308} gitlab # [ 246.940924] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.189Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"223d05f9e372392d119b1cf4","created_at":"2026-09-23T19:21:18.026Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.035Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-223d05f9e372392d119b1cf4: done: 0.024288 sec","job_status":"done","queue_duration_s":2.130294,"scheduling_latency_s":2.130308,"redis_calls":10,"redis_duration_s":0.002082,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000377,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001247,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000458,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000283276,"duration_s":0.024288,"completed_at":"2026-09-23T19:21:20.189Z","load_balancing_strategy":"primary","db_duration_s":0.006522,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 246.982691] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.191Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"92a5e2eb060691aa5db16917","created_at":"2026-09-23T19:21:18.048Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.051Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-92a5e2eb060691aa5db16917: start","job_status":"start","queue_duration_s":2.139287,"scheduling_latency_s":2.139301} gitlab # [ 246.985763] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.216Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"92a5e2eb060691aa5db16917","created_at":"2026-09-23T19:21:18.048Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.051Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-92a5e2eb060691aa5db16917: done: 0.024926 sec","job_status":"done","queue_duration_s":2.139287,"scheduling_latency_s":2.139301,"redis_calls":13,"redis_duration_s":0.00268,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000398,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001216,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000651,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000415,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000319035,"duration_s":0.024926,"completed_at":"2026-09-23T19:21:20.216Z","load_balancing_strategy":"primary","db_duration_s":0.005411,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.025281] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.217Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"0c54d27c20482d20cd5bffd8","created_at":"2026-09-23T19:21:18.052Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.112Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-0c54d27c20482d20cd5bffd8: start","job_status":"start","queue_duration_s":2.104959,"scheduling_latency_s":2.104974} gitlab # [ 247.063825] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.238Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"0c54d27c20482d20cd5bffd8","created_at":"2026-09-23T19:21:18.052Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.112Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-0c54d27c20482d20cd5bffd8: done: 0.021013 sec","job_status":"done","queue_duration_s":2.104959,"scheduling_latency_s":2.104974,"redis_calls":10,"redis_duration_s":0.002087,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000425,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001234,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000269308,"duration_s":0.021013,"completed_at":"2026-09-23T19:21:20.238Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003978,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.102756] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.239Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"8fe94a65a40d1da195a69ac7","created_at":"2026-09-23T19:21:18.113Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.116Z","job_size_bytes":19,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-8fe94a65a40d1da195a69ac7: start","job_status":"start","queue_duration_s":2.122831,"scheduling_latency_s":2.122844} gitlab # [ 247.103361] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.258Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"8fe94a65a40d1da195a69ac7","created_at":"2026-09-23T19:21:18.113Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.116Z","job_size_bytes":19,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-8fe94a65a40d1da195a69ac7: done: 0.019456 sec","job_status":"done","queue_duration_s":2.122831,"scheduling_latency_s":2.122844,"redis_calls":11,"redis_duration_s":0.0021520000000000003,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000369,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001356,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000253664,"duration_s":0.019456,"completed_at":"2026-09-23T19:21:20.258Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003857,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.142925] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.260Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"45df43e762f4a10b44de6e56","created_at":"2026-09-23T19:21:18.118Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.121Z","job_size_bytes":66,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-45df43e762f4a10b44de6e56: start","job_status":"start","queue_duration_s":2.138733,"scheduling_latency_s":2.138745} gitlab # [ 247.145534] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.315Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"45df43e762f4a10b44de6e56","created_at":"2026-09-23T19:21:18.118Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.121Z","job_size_bytes":66,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-45df43e762f4a10b44de6e56: done: 0.055057 sec","job_status":"done","queue_duration_s":2.138733,"scheduling_latency_s":2.138745,"redis_calls":10,"redis_duration_s":0.002075,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000522,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001147,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000406,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.036,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000325461,"duration_s":0.055057,"completed_at":"2026-09-23T19:21:20.315Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.036768,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.203294] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.316Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"19dd033c91401a3ace1e45ec","created_at":"2026-09-23T19:21:18.122Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.124Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-19dd033c91401a3ace1e45ec: start","job_status":"start","queue_duration_s":2.191745,"scheduling_latency_s":2.19176} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1' gitlab # [ 247.205911] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.344Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"19dd033c91401a3ace1e45ec","created_at":"2026-09-23T19:21:18.122Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.124Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-19dd033c91401a3ace1e45ec: done: 0.028363 sec","job_status":"done","queue_duration_s":2.191745,"scheduling_latency_s":2.19176,"redis_calls":10,"redis_duration_s":0.00179,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000244,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001099,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000447,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000764064,"duration_s":0.028363,"completed_at":"2026-09-23T19:21:20.344Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00282,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1', in 0.03 seconds) gitlab # [ 247.249988] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.346Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"9f272781842541fa66d8a1fd","created_at":"2026-09-23T19:21:18.128Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.133Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9f272781842541fa66d8a1fd: start","job_status":"start","queue_duration_s":2.213249,"scheduling_latency_s":2.213283} gitlab # [ 247.287794] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.393Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"9f272781842541fa66d8a1fd","created_at":"2026-09-23T19:21:18.128Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.133Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-9f272781842541fa66d8a1fd: done: 0.046324 sec","job_status":"done","queue_duration_s":2.213249,"scheduling_latency_s":2.213283,"redis_calls":10,"redis_duration_s":0.002088,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000381,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001294,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000413,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029848,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000430502,"duration_s":0.046324,"completed_at":"2026-09-23T19:21:20.393Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007572,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.327047] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.394Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"50ad1fc20e7b7a53b254b0bd","created_at":"2026-09-23T19:21:18.134Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.141Z","job_size_bytes":6,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-50ad1fc20e7b7a53b254b0bd: start","job_status":"start","queue_duration_s":2.253071,"scheduling_latency_s":2.253097} gitlab # [ 247.327717] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.413Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"50ad1fc20e7b7a53b254b0bd","created_at":"2026-09-23T19:21:18.134Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.141Z","job_size_bytes":6,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-50ad1fc20e7b7a53b254b0bd: done: 0.019116 sec","job_status":"done","queue_duration_s":2.253071,"scheduling_latency_s":2.253097,"redis_calls":10,"redis_duration_s":0.001488,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000249,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000384,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000227123,"duration_s":0.019116,"completed_at":"2026-09-23T19:21:20.413Z","load_balancing_strategy":"primary","db_duration_s":0.00134,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.343949] nixbot[2658]: INFO: ::ffff:127.0.0.1:52690 - "GET /api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1 HTTP/1.1" 200 OK gitlab # [ 247.381478] gitaly[890]: time="2026-09-23T19:21:20.539Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ diskcache=cc78f352-7a78-440c-9da0-d79524ac81a0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-24T01:21:20.524" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:20.524" pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.382191] gitaly[890]: time="2026-09-23T19:21:20.539Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-24T01:21:20.524" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:20.524" grpc.time_ms=14.667 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.382775] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.414Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"d067e0cbf3b82c918a3e262c","created_at":"2026-09-23T19:21:18.142Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.146Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-d067e0cbf3b82c918a3e262c: start","job_status":"start","queue_duration_s":2.267741,"scheduling_latency_s":2.267757} gitlab # [ 247.383508] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.437Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"d067e0cbf3b82c918a3e262c","created_at":"2026-09-23T19:21:18.142Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.146Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-d067e0cbf3b82c918a3e262c: done: 0.023087 sec","job_status":"done","queue_duration_s":2.267741,"scheduling_latency_s":2.267757,"redis_calls":10,"redis_duration_s":0.001238,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000235,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000191365,"duration_s":0.023087,"completed_at":"2026-09-23T19:21:20.437Z","load_balancing_strategy":"primary","db_duration_s":0.004164,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.415174] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.438Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"ea52afeb02d304b2f32476d0","created_at":"2026-09-23T19:21:18.152Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.155Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-ea52afeb02d304b2f32476d0: start","job_status":"start","queue_duration_s":2.283588,"scheduling_latency_s":2.283601} gitlab # [ 247.459221] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.483Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"ea52afeb02d304b2f32476d0","created_at":"2026-09-23T19:21:18.152Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.155Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-ea52afeb02d304b2f32476d0: done: 0.044397 sec","job_status":"done","queue_duration_s":2.283588,"scheduling_latency_s":2.283601,"redis_calls":14,"redis_duration_s":0.005702,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000241,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004339,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001122,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000316521,"duration_s":0.044397,"completed_at":"2026-09-23T19:21:20.483Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009869,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.501837] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.484Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"d90a0259e53beca0b9285ce8","created_at":"2026-09-23T19:21:18.218Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.220Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d90a0259e53beca0b9285ce8: start","job_status":"start","queue_duration_s":2.263729,"scheduling_latency_s":2.263744} gitlab # [ 247.502565] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.503Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"d90a0259e53beca0b9285ce8","created_at":"2026-09-23T19:21:18.218Z","trace_propagation_headers":{"sentry-trace":"c1031c8acc254914a3dcdde36c696e53-ccde8be679ba460a","baggage":"sentry-trace_id=c1031c8acc254914a3dcdde36c696e53,sentry-release=Unknown"},"correlation_id":"01M37VCX7669597E0KHBJ945B0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.220Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-d90a0259e53beca0b9285ce8: done: 0.019901 sec","job_status":"done","queue_duration_s":2.263729,"scheduling_latency_s":2.263744,"redis_calls":8,"redis_duration_s":0.0011639999999999999,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019901,"completed_at":"2026-09-23T19:21:20.503Z","load_balancing_strategy":"primary","db_duration_s":0.003778,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 247.515726] gitaly[890]: time="2026-09-23T19:21:20.695Z" level=info msg="finished unary call with code OK" catfile.duration_ms=77 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=77 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:50.468" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:20.617" grpc.time_ms=77.859 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.556115] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.505Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8752d97574ca07f96174c76e","created_at":"2026-09-23T19:21:18.350Z","trace_propagation_headers":{"sentry-trace":"ceb0b6f3e456474babb7761654ca1b6a-dc1a53d4b9f449e0","baggage":"sentry-trace_id=ceb0b6f3e456474babb7761654ca1b6a,sentry-release=Unknown"},"correlation_id":"01M37VE2Q2PTE6S3JA2024M7JJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"4154acce85a438280f8d0a5e","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.352Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8752d97574ca07f96174c76e: start","job_status":"start","queue_duration_s":2.15288,"scheduling_latency_s":2.152901} gitlab # [ 247.556803] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.554Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8752d97574ca07f96174c76e","created_at":"2026-09-23T19:21:18.350Z","trace_propagation_headers":{"sentry-trace":"ceb0b6f3e456474babb7761654ca1b6a-dc1a53d4b9f449e0","baggage":"sentry-trace_id=ceb0b6f3e456474babb7761654ca1b6a,sentry-release=Unknown"},"correlation_id":"01M37VE2Q2PTE6S3JA2024M7JJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"4154acce85a438280f8d0a5e","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.352Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8752d97574ca07f96174c76e: done: 0.049212 sec","job_status":"done","queue_duration_s":2.15288,"scheduling_latency_s":2.152901,"gitaly_calls":1,"gitaly_duration_s":0.016692,"redis_calls":12,"redis_duration_s":0.002655,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001197,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000369,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001089,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.029484194,"duration_s":0.049212,"completed_at":"2026-09-23T19:21:20.554Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004415,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.612771] gitaly[890]: time="2026-09-23T19:21:20.703Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=625 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-23T19:21:30.008" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:20.698" grpc.time_ms=5.001 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.613623] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:20.555Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"60a1e224dc9247621f41f562","created_at":"2026-09-23T19:21:18.503Z","trace_propagation_headers":{"sentry-trace":"ceb0b6f3e456474babb7761654ca1b6a-dc1a53d4b9f449e0","baggage":"sentry-trace_id=ceb0b6f3e456474babb7761654ca1b6a,sentry-release=Unknown"},"correlation_id":"01M37VE2Q2PTE6S3JA2024M7JJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cb0ff15e4e9e7e55b448ba218c3ff5b78b89079a267944ce90c55eeca1344fd4","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.505Z","job_size_bytes":467,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-60a1e224dc9247621f41f562: start","job_status":"start","queue_duration_s":2.049701,"scheduling_latency_s":2.049718} gitlab # [ 247.614247] gitaly[890]: time="2026-09-23T19:21:20.724Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-23T19:21:50.623" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:20.723" grpc.time_ms=0.888 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.620638] gitaly[890]: time="2026-09-23T19:21:20.728Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=44 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-23T19:21:50.626" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:20.726" grpc.time_ms=1.595 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.635645] gitaly[890]: time="2026-09-23T19:21:20.749Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=0 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:21:30.054" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:20.744" grpc.time_ms=4.808 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.735543] gitaly[890]: time="2026-09-23T19:21:21.008Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=630 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=3 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VE8T7R5H1X89A9VFY495F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:21.001" grpc.time_ms=6.742 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 247.747771] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:21 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 42 gitlab # [ 247.789279] gitaly[890]: time="2026-09-23T19:21:21.061Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VE8VKVNNSE5DEDGVY8FAA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:21.054" pid=890 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 247.798801] gitaly[890]: time="2026-09-23T19:21:21.062Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE8VKVNNSE5DEDGVY8FAA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:21.054" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 247.799127] gitlab-workhorse[825]: time="2026-09-23T19:21:21Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 247.799588] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:21 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 52 gitlab # [ 247.800123] gitaly[890]: time="2026-09-23T19:21:21.063Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=642 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE8VKVNNSE5DEDGVY8FAA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:21.054" grpc.time_ms=8.535 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 247.822202] gitaly[890]: time="2026-09-23T19:21:21.094Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=962 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-23T19:21:51.085" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:21.085" grpc.time_ms=9.231 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.841708] gitaly[890]: time="2026-09-23T19:21:21.114Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VE8X9SY2ZCHA3HE65V0B6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:21.104" pid=890 remote_ip="2001:db8:1::1" request_sha=45918087779c16b09c11e58e8b2311a8d2ef31ba response_bytes=336 span.kind=server system=grpc gitlab # [ 247.842072] gitaly[890]: time="2026-09-23T19:21:21.114Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1190 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE8X9SY2ZCHA3HE65V0B6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:21.104" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 247.843144] gitaly[890]: time="2026-09-23T19:21:21.114Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1190 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE8X9SY2ZCHA3HE65V0B6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:21.104" grpc.time_ms=10.001 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 247.843934] gitlab-workhorse[825]: time="2026-09-23T19:21:21Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=336 gitlab # [ 247.844511] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:21 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.55.0" 50 gitlab # [ 247.884717] gitaly[890]: time="2026-09-23T19:21:21.157Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=37 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=629 command.oublock=0 command.real_time_ms=56 command.spawn_token_fork_ms=15 command.system_time_ms=37 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VE2Q2PTE6S3JA2024M7JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-23T19:21:51.097" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:21.097" grpc.time_ms=59.878 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 247.896044] sidekiq-cluster[2530]: {"severity":"WARN","time":"2026-09-23T19:21:21.167Z","class":"CreatePipelineWorker","correlation_id":"01M37VE2Q2PTE6S3JA2024M7JJ","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0a686633e096d704cae69e46c6345dd0bcf8d3c3","after":"263054a42f00ab5c2554e36527725bb48ab5e4c1","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"263054a42f00ab5c2554e36527725bb48ab5e4c1","ref":"refs/heads/master"},"retry":0} gitlab # [ 247.901114] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.173Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"60a1e224dc9247621f41f562","created_at":"2026-09-23T19:21:18.503Z","trace_propagation_headers":{"sentry-trace":"ceb0b6f3e456474babb7761654ca1b6a-dc1a53d4b9f449e0","baggage":"sentry-trace_id=ceb0b6f3e456474babb7761654ca1b6a,sentry-release=Unknown"},"correlation_id":"01M37VE2Q2PTE6S3JA2024M7JJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cb0ff15e4e9e7e55b448ba218c3ff5b78b89079a267944ce90c55eeca1344fd4","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.505Z","job_size_bytes":467,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-60a1e224dc9247621f41f562: done: 0.618439 sec","job_status":"done","queue_duration_s":2.049701,"scheduling_latency_s":2.049718,"gitaly_calls":7,"gitaly_duration_s":0.220199,"redis_calls":33,"redis_duration_s":0.004909,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000348,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002213,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001579,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000769,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.383626,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.618439,"completed_at":"2026-09-23T19:21:21.173Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010903,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 247.906763] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.174Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"7c45c1d2-0500-4342-8f1c-d6019eea2f97\"}"],"class":"WebHookWorker","jid":"9d72039912fb6b917c6377cc","created_at":"2026-09-23T19:21:18.523Z","trace_propagation_headers":{"sentry-trace":"ceb0b6f3e456474babb7761654ca1b6a-dc1a53d4b9f449e0","baggage":"sentry-trace_id=ceb0b6f3e456474babb7761654ca1b6a,sentry-release=Unknown"},"correlation_id":"01M37VE2Q2PTE6S3JA2024M7JJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0f33e7cfb3ff8c63c4f409ed6aac6b31ccc7b18fd0b123cf68d6c645a87dee9e","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.526Z","job_size_bytes":1749,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-9d72039912fb6b917c6377cc: start","job_status":"start","queue_duration_s":2.648535,"scheduling_latency_s":2.648581} gitlab # [ 247.949218] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.201Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"7c45c1d2-0500-4342-8f1c-d6019eea2f97\"}"],"class":"WebHookWorker","jid":"9d72039912fb6b917c6377cc","created_at":"2026-09-23T19:21:18.523Z","trace_propagation_headers":{"sentry-trace":"ceb0b6f3e456474babb7761654ca1b6a-dc1a53d4b9f449e0","baggage":"sentry-trace_id=ceb0b6f3e456474babb7761654ca1b6a,sentry-release=Unknown"},"correlation_id":"01M37VE2Q2PTE6S3JA2024M7JJ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0f33e7cfb3ff8c63c4f409ed6aac6b31ccc7b18fd0b123cf68d6c645a87dee9e","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.526Z","job_size_bytes":1749,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-9d72039912fb6b917c6377cc: done: 0.026118 sec","job_status":"done","queue_duration_s":2.648535,"scheduling_latency_s":2.648581,"redis_calls":14,"redis_duration_s":0.001774,"redis_read_bytes":25,"redis_write_bytes":4683,"redis_queues_calls":2,"redis_queues_duration_s":0.000323,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3727,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000512,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.004890007000000196,"cpu_s":0.016082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.026118,"completed_at":"2026-09-23T19:21:21.200Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001173,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.964395] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.202Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"2e04f38605818bda347a2700","created_at":"2026-09-23T19:21:18.778Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-42634f4d3b3446a3","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.784Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-2e04f38605818bda347a2700: start","job_status":"start","queue_duration_s":2.417358,"scheduling_latency_s":2.41738} gitlab # [ 248.005873] nixbot[2658]: INFO: ::ffff:127.0.0.1:52702 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 248.006744] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.232Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"2e04f38605818bda347a2700","created_at":"2026-09-23T19:21:18.778Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-42634f4d3b3446a3","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:18.784Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-2e04f38605818bda347a2700: done: 0.030342 sec","job_status":"done","queue_duration_s":2.417358,"scheduling_latency_s":2.41738,"redis_calls":19,"redis_duration_s":0.003003,"redis_read_bytes":21,"redis_write_bytes":4292,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000992,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000333,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2263,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001098,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00058,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.005154287,"duration_s":0.030342,"completed_at":"2026-09-23T19:21:21.232Z","load_balancing_strategy":"primary","db_duration_s":0.004535,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.050868] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.233Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","a63b4a459b7b76a1e9d651db"],"class":"WebHooks::LogExecutionWorker","jid":"9b1618a16cf08e0278e4e316","created_at":"2026-09-23T19:21:20.069Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-92780fbe8fca42ca","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4af28279bd7b21c52a9faac4085c80fd9e8b6f6027352bfe93d3175f7ff3ea59","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:20.074Z","job_size_bytes":2593,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-9b1618a16cf08e0278e4e316: start","job_status":"start","queue_duration_s":1.158982,"scheduling_latency_s":1.158996} gitlab # [ 248.051808] gitaly[890]: time="2026-09-23T19:21:21.305Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=635 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VE93H9J9RET3HB6VASX0F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:21.298" grpc.time_ms=5.826 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 248.052531] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:21 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 40 gitlab # [ 248.054735] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.262Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","a63b4a459b7b76a1e9d651db"],"class":"WebHooks::LogExecutionWorker","jid":"9b1618a16cf08e0278e4e316","created_at":"2026-09-23T19:21:20.069Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-92780fbe8fca42ca","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4af28279bd7b21c52a9faac4085c80fd9e8b6f6027352bfe93d3175f7ff3ea59","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:20.074Z","job_size_bytes":2593,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-9b1618a16cf08e0278e4e316: done: 0.029071 sec","job_status":"done","queue_duration_s":1.158982,"scheduling_latency_s":1.158996,"redis_calls":10,"redis_duration_s":0.0013109999999999999,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000797,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000258,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000195276,"duration_s":0.029071,"completed_at":"2026-09-23T19:21:21.262Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00636,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.109894] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.263Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","9d72039912fb6b917c6377cc"],"class":"WebHooks::LogExecutionWorker","jid":"0b88b4724c100a3eb8453b65","created_at":"2026-09-23T19:21:21.194Z","trace_propagation_headers":{"sentry-trace":"ceb0b6f3e456474babb7761654ca1b6a-c5fe76ad2a084e4b","baggage":"sentry-trace_id=ceb0b6f3e456474babb7761654ca1b6a,sentry-release=Unknown"},"correlation_id":"01M37VE2Q2PTE6S3JA2024M7JJ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d7fffb124177ef88a948ac619f0c24a75868e1e688a99609c871ed02d0b917db","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:21.195Z","job_size_bytes":2590,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-0b88b4724c100a3eb8453b65: start","job_status":"start","queue_duration_s":0.067936,"scheduling_latency_s":0.067949} gitlab # [ 248.110778] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.290Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","9d72039912fb6b917c6377cc"],"class":"WebHooks::LogExecutionWorker","jid":"0b88b4724c100a3eb8453b65","created_at":"2026-09-23T19:21:21.194Z","trace_propagation_headers":{"sentry-trace":"ceb0b6f3e456474babb7761654ca1b6a-c5fe76ad2a084e4b","baggage":"sentry-trace_id=ceb0b6f3e456474babb7761654ca1b6a,sentry-release=Unknown"},"correlation_id":"01M37VE2Q2PTE6S3JA2024M7JJ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d7fffb124177ef88a948ac619f0c24a75868e1e688a99609c871ed02d0b917db","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:21.195Z","job_size_bytes":2590,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-0b88b4724c100a3eb8453b65: done: 0.027057 sec","job_status":"done","queue_duration_s":0.067936,"scheduling_latency_s":0.067949,"redis_calls":7,"redis_duration_s":0.001216,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000817,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000399,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027057,"completed_at":"2026-09-23T19:21:21.290Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006421,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.164628] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.291Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"ebe67172e32d9783b7f3d81d","created_at":"2026-09-23T19:21:21.223Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-27e402b68b6e40ef","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:21.227Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-ebe67172e32d9783b7f3d81d: start","job_status":"start","queue_duration_s":0.064362,"scheduling_latency_s":0.064379} gitlab # [ 248.165425] gitlab-workhorse[825]: time="2026-09-23T19:21:21Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 248.165898] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:21 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 37 gitlab # [ 248.166352] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:21 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 37 gitlab # [ 248.166914] gitaly[890]: time="2026-09-23T19:21:21.343Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VE94T7Q34PEXG8R7ED6R3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:21.337" pid=890 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 248.171447] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.355Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"ebe67172e32d9783b7f3d81d","created_at":"2026-09-23T19:21:21.223Z","trace_propagation_headers":{"sentry-trace":"87a12dfebda0472fb329e33b09e1a097-27e402b68b6e40ef","baggage":"sentry-trace_id=87a12dfebda0472fb329e33b09e1a097,sentry-release=Unknown"},"correlation_id":"01M37VCRKKAV8H11BT0FYE2J2X","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:21.227Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-ebe67172e32d9783b7f3d81d: done: 0.063894 sec","job_status":"done","queue_duration_s":0.064362,"scheduling_latency_s":0.064379,"redis_calls":11,"redis_duration_s":0.0029430000000000003,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000701,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000959,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001283,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000223213,"duration_s":0.063894,"completed_at":"2026-09-23T19:21:21.355Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018805,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.219173] gitaly[890]: time="2026-09-23T19:21:21.343Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE94T7Q34PEXG8R7ED6R3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:21.337" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 248.223888] gitlab-workhorse[825]: time="2026-09-23T19:21:21Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 248.225557] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:21 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 40 gitlab # [ 248.227199] gitaly[890]: time="2026-09-23T19:21:21.343Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=638 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE94T7Q34PEXG8R7ED6R3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:21.337" grpc.time_ms=6.423 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1' gitlab # [ 248.240056] gitaly[890]: time="2026-09-23T19:21:21.408Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=631 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VE96TXDXGE5C3VE68A1FM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:21.402" grpc.time_ms=5.613 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 248.282924] gitaly[890]: time="2026-09-23T19:21:21.449Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M37VE981XY411ZKYCBZQY0KF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:21.438" pid=890 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 248.283609] gitaly[890]: time="2026-09-23T19:21:21.449Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE981XY411ZKYCBZQY0KF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:21.438" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1', in 0.03 seconds) (finished: subtest: push delivers the webhook, in 46.57 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses gitlab # [ 248.284200] gitaly[890]: time="2026-09-23T19:21:21.449Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1168 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE981XY411ZKYCBZQY0KF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-23T19:21:21.438" grpc.time_ms=10.97 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 248.285000] gitaly[890]: time="2026-09-23T19:21:21.517Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE9AANPWWJTQV8Q4BBPTF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:51.416" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:21.516" grpc.time_ms=0.472 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.328871] gitaly[890]: time="2026-09-23T19:21:21.528Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=647 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VE9AANPWWJTQV8Q4BBPTF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-23T19:21:51.422" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:21.522" grpc.time_ms=5.681 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.329596] nixbot[2658]: INFO: ::ffff:127.0.0.1:52710 - "GET /api/repos/gitlab/root/test-flake/builds?commit=263054a42f00ab5c2554e36527725bb48ab5e4c1 HTTP/1.1" 200 OK gitlab # [ 248.330064] gitaly[890]: time="2026-09-23T19:21:21.536Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=627 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VE9AANPWWJTQV8Q4BBPTF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:21:31.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:21.531" grpc.time_ms=4.441 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.330669] gitaly[890]: time="2026-09-23T19:21:21.548Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M37VE9AANPWWJTQV8Q4BBPTF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-23T19:21:31.017" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:21:21.548" grpc.time_ms=0.146 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.331650] gitaly[890]: time="2026-09-23T19:21:21.558Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE9AANPWWJTQV8Q4BBPTF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-23T19:21:31.019" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:21.550" grpc.time_ms=7.974 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.343433] gitaly[890]: time="2026-09-23T19:21:21.589Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VE9AANPWWJTQV8Q4BBPTF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4150 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:21.588" grpc.time_ms=1.343 pid=890 span.kind=server system=grpc gitlab # [ 248.384741] gitaly[890]: time="2026-09-23T19:21:21.612Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VE9AANPWWJTQV8Q4BBPTF grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:21.612" grpc.time_ms=0.245 pid=890 span.kind=server system=grpc gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses, in 0.09 seconds) gitlab # [ 248.385542] gitaly[890]: time="2026-09-23T19:21:21.624Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VE9AANPWWJTQV8Q4BBPTF diskcache=f78c397a-0e93-4a53-a94c-1cd3f1778e4f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-23T19:21:31.033" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:21:21.563" pid=890 ref=refs/keep-around/263054a42f00ab5c2554e36527725bb48ab5e4c1 remote_ip="2001:db8:1::1" revision=263054a42f00ab5c2554e36527725bb48ab5e4c1 span.kind=server system=grpc username=root gitlab # [ 248.387786] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.635Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ece0f5859b6006f0b291a8a1","created_at":"2026-09-23T19:21:21.629Z","trace_propagation_headers":{"sentry-trace":"1659b99f17254f99a0cbd44bb1587369-0ded0d4bb05947b3","baggage":"sentry-trace_id=1659b99f17254f99a0cbd44bb1587369,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VE9AANPWWJTQV8Q4BBPTF","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:21.632Z","job_size_bytes":65,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ece0f5859b6006f0b291a8a1: start","job_status":"start","queue_duration_s":0.003093,"scheduling_latency_s":0.003123} gitlab # [ 248.391148] gitaly[890]: time="2026-09-23T19:21:21.624Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=42 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5211 command.oublock=8 command.real_time_ms=53 command.spawn_token_fork_ms=0 command.system_time_ms=31 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE9AANPWWJTQV8Q4BBPTF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-23T19:21:31.033" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-23T19:21:21.563" grpc.time_ms=61.007 pid=890 ref=refs/keep-around/263054a42f00ab5c2554e36527725bb48ab5e4c1 remote_ip="2001:db8:1::1" revision=263054a42f00ab5c2554e36527725bb48ab5e4c1 span.kind=server system=grpc username=root gitlab # [ 248.398914] gitaly[890]: time="2026-09-23T19:21:21.628Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE9AANPWWJTQV8Q4BBPTF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:51.526" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:21.627" grpc.time_ms=0.693 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.439589] nixbot[2658]: {"ts": "2026-09-23T19:21:21.666Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 248.440670] nixbot[2658]: {"ts": "2026-09-23T19:21:21.666Z", "level": "info", "logger": "nixbot.canceller", "msg": "ignoring out-of-order change event", "project_id": 1, "context": "branch/master", "stale_commit": "0a686633e096d704cae69e46c6345dd0bcf8d3c3"} gitlab # [ 248.441523] nixbot[2658]: {"ts": "2026-09-23T19:21:21.706Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/263054a42f00ab5c2554e36527725bb48ab5e4c1 \"HTTP/1.1 201 Created\""} gitlab # [ 248.442001] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.675Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ece0f5859b6006f0b291a8a1","created_at":"2026-09-23T19:21:21.629Z","trace_propagation_headers":{"sentry-trace":"1659b99f17254f99a0cbd44bb1587369-0ded0d4bb05947b3","baggage":"sentry-trace_id=1659b99f17254f99a0cbd44bb1587369,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VE9AANPWWJTQV8Q4BBPTF","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:21.632Z","job_size_bytes":65,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ece0f5859b6006f0b291a8a1: done: 0.040517 sec","job_status":"done","queue_duration_s":0.003093,"scheduling_latency_s":0.003123,"redis_calls":7,"redis_duration_s":0.0055320000000000005,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004426,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001106,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040517,"completed_at":"2026-09-23T19:21:21.675Z","load_balancing_strategy":"primary","db_duration_s":0.008294,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 248.487136] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.679Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c763fb1117036d001f861207","created_at":"2026-09-23T19:21:21.633Z","trace_propagation_headers":{"sentry-trace":"1659b99f17254f99a0cbd44bb1587369-0ded0d4bb05947b3","baggage":"sentry-trace_id=1659b99f17254f99a0cbd44bb1587369,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VE9AANPWWJTQV8Q4BBPTF","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:21.636Z","job_size_bytes":65,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c763fb1117036d001f861207: start","job_status":"start","queue_duration_s":0.04297,"scheduling_latency_s":0.042989} gitlab # [ 248.487872] gitaly[890]: time="2026-09-23T19:21:21.644Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE9DZT18XNTQWGAFFT9RC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:51.143" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:21.644" grpc.time_ms=0.461 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.488501] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:21 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses HTTP/1.1" 200 2 "" "curl/8.21.0" 76 gitlab # [ 248.489397] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:21 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/263054a42f00ab5c2554e36527725bb48ab5e4c1 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 222 gitlab # [ 248.490026] nixbot[2658]: {"ts": "2026-09-23T19:21:21.714Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-263054a42f00-4"} gitlab # [ 248.490505] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.725Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c763fb1117036d001f861207","created_at":"2026-09-23T19:21:21.633Z","trace_propagation_headers":{"sentry-trace":"1659b99f17254f99a0cbd44bb1587369-0ded0d4bb05947b3","baggage":"sentry-trace_id=1659b99f17254f99a0cbd44bb1587369,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VE9AANPWWJTQV8Q4BBPTF","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:21.636Z","job_size_bytes":65,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c763fb1117036d001f861207: done: 0.046669 sec","job_status":"done","queue_duration_s":0.04297,"scheduling_latency_s":0.042989,"redis_calls":10,"redis_duration_s":0.002479,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001545,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046669,"completed_at":"2026-09-23T19:21:21.725Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014944,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.531089] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.726Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"67a123132cd153577a610647","created_at":"2026-09-23T19:21:21.692Z","trace_propagation_headers":{"sentry-trace":"1659b99f17254f99a0cbd44bb1587369-0ded0d4bb05947b3","baggage":"sentry-trace_id=1659b99f17254f99a0cbd44bb1587369,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VE9AANPWWJTQV8Q4BBPTF","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:21.694Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-67a123132cd153577a610647: start","job_status":"start","queue_duration_s":0.032012,"scheduling_latency_s":0.03203} gitlab # [ 248.537900] gitaly[890]: time="2026-09-23T19:21:21.774Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VE9AANPWWJTQV8Q4BBPTF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-23T19:21:51.774" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:21.774" grpc.time_ms=0.616 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.549198] nixbot[2658]: {"ts": "2026-09-23T19:21:21.804Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-263054a42f00-4", "argv0": "bwrap"} gitlab # [ 248.551470] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.810Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"67a123132cd153577a610647","created_at":"2026-09-23T19:21:21.692Z","trace_propagation_headers":{"sentry-trace":"1659b99f17254f99a0cbd44bb1587369-0ded0d4bb05947b3","baggage":"sentry-trace_id=1659b99f17254f99a0cbd44bb1587369,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VE9AANPWWJTQV8Q4BBPTF","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:21.694Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-67a123132cd153577a610647: done: 0.084206 sec","job_status":"done","queue_duration_s":0.032012,"scheduling_latency_s":0.03203,"gitaly_calls":2,"gitaly_duration_s":0.005191,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.005444,"redis_read_bytes":79,"redis_write_bytes":5120,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000658,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000727,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000401,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1131,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001901,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000165,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000343,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000591,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053549,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.063429849,"duration_s":0.084206,"completed_at":"2026-09-23T19:21:21.810Z","load_balancing_strategy":"primary","db_duration_s":0.022829,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 248.597543] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.812Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d8b9d9732c9cdaeb6ec1b6ac","created_at":"2026-09-23T19:21:21.763Z","trace_propagation_headers":{"sentry-trace":"1659b99f17254f99a0cbd44bb1587369-d3f72acc37d94d0c","baggage":"sentry-trace_id=1659b99f17254f99a0cbd44bb1587369,sentry-release=Unknown"},"correlation_id":"01M37VE9AANPWWJTQV8Q4BBPTF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:21.765Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d8b9d9732c9cdaeb6ec1b6ac: start","job_status":"start","queue_duration_s":0.046419,"scheduling_latency_s":0.046436} gitlab # [ 248.640676] gitaly[890]: time="2026-09-23T19:21:21.800Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VE9AANPWWJTQV8Q4BBPTF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:51.799" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:21.799" grpc.time_ms=0.515 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 248.641604] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:21.844Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"d8b9d9732c9cdaeb6ec1b6ac","created_at":"2026-09-23T19:21:21.763Z","trace_propagation_headers":{"sentry-trace":"1659b99f17254f99a0cbd44bb1587369-d3f72acc37d94d0c","baggage":"sentry-trace_id=1659b99f17254f99a0cbd44bb1587369,sentry-release=Unknown"},"correlation_id":"01M37VE9AANPWWJTQV8Q4BBPTF","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:21.765Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-d8b9d9732c9cdaeb6ec1b6ac: done: 0.032172 sec","job_status":"done","queue_duration_s":0.046419,"scheduling_latency_s":0.046436,"redis_calls":7,"redis_duration_s":0.001081,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000799,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000282,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032172,"completed_at":"2026-09-23T19:21:21.844Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0143,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.748534] nix-daemon[2823]: accepted connection from pid 3574, user nixbot gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses gitlab # [ 249.508955] gitaly[890]: time="2026-09-23T19:21:22.780Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEAGZDR1HG71DW84AHNXH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:52.279" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:22.779" grpc.time_ms=1.026 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 249.533682] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 86 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses, in 0.14 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses gitlab # [ 250.635084] gitaly[890]: time="2026-09-23T19:21:23.907Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEBM9PXT2QKMP6KZDZRFG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:53.105" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:23.906" grpc.time_ms=0.865 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 250.664789] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 87 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses, in 0.13 seconds) gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses gitlab # [ 251.799619] gitaly[890]: time="2026-09-23T19:21:25.072Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VECQKEHM1ZZZQRFHQMAJ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:55.070" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:25.070" grpc.time_ms=1.043 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 251.835427] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 128 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses, in 0.17 seconds) gitlab # [ 252.474523] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:25.746Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"fbbbc371176fd657f6f2410d","created_at":"2026-09-23T19:21:25.729Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2cb35b79060b81c7fade27a2e0bb3eae","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:25.743Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-fbbbc371176fd657f6f2410d: start","job_status":"start","queue_duration_s":0.002297,"scheduling_latency_s":0.002336} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses gitlab # [ 252.923137] gitaly[890]: time="2026-09-23T19:21:26.195Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEDW42DBE98J388GM7JPQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:56.194" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:26.194" grpc.time_ms=0.632 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 252.943500] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:26 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 66 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses, in 0.10 seconds) gitlab # [ 253.281535] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.553Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"fbbbc371176fd657f6f2410d","created_at":"2026-09-23T19:21:25.729Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2cb35b79060b81c7fade27a2e0bb3eae","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:25.743Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-fbbbc371176fd657f6f2410d: done: 0.807588 sec","job_status":"done","queue_duration_s":0.002297,"scheduling_latency_s":0.002336,"redis_calls":564,"redis_duration_s":0.442664,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002069,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004223,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.436372,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.381228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.807588,"completed_at":"2026-09-23T19:21:26.553Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002762,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.320956] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.593Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f2f8c03999c7b89c2a1fde3f","created_at":"2026-09-23T19:21:25.811Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0b26ce39921849d072f3c096c21245c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:25.847Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f2f8c03999c7b89c2a1fde3f: start","job_status":"start","queue_duration_s":0.745638,"scheduling_latency_s":0.745694} gitlab # [ 253.340159] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.612Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 253.351527] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.623Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"f2f8c03999c7b89c2a1fde3f","created_at":"2026-09-23T19:21:25.811Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f0b26ce39921849d072f3c096c21245c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:25.847Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-f2f8c03999c7b89c2a1fde3f: done: 0.030684 sec","job_status":"done","queue_duration_s":0.745638,"scheduling_latency_s":0.745694,"redis_calls":9,"redis_duration_s":0.002967,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000629,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001545,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000793,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030684,"completed_at":"2026-09-23T19:21:26.623Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000965,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.353474] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.626Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"d5b30373bfa58052d2b19a9a","created_at":"2026-09-23T19:21:25.857Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11d7645faaabc66c4f342bc77c1ad9ac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:25.867Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-d5b30373bfa58052d2b19a9a: start","job_status":"start","queue_duration_s":0.758094,"scheduling_latency_s":0.758128} gitlab # [ 253.385659] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.658Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"d5b30373bfa58052d2b19a9a","created_at":"2026-09-23T19:21:25.857Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"11d7645faaabc66c4f342bc77c1ad9ac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:25.867Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-d5b30373bfa58052d2b19a9a: done: 0.032364 sec","job_status":"done","queue_duration_s":0.758094,"scheduling_latency_s":0.758128,"redis_calls":18,"redis_duration_s":0.004507,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001004,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000296,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002507,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.0007,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032364,"completed_at":"2026-09-23T19:21:26.658Z","load_balancing_strategy":"primary","db_duration_s":0.002938,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.400424] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.659Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"cf8d9818409d42104b1004f6","created_at":"2026-09-23T19:21:25.878Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae1f1788e20ff06921093ee9907eb117","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:25.889Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-cf8d9818409d42104b1004f6: start","job_status":"start","queue_duration_s":0.769621,"scheduling_latency_s":0.769636} gitlab # [ 253.413623] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.679Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"cf8d9818409d42104b1004f6","created_at":"2026-09-23T19:21:25.878Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae1f1788e20ff06921093ee9907eb117","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:25.889Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-cf8d9818409d42104b1004f6: done: 0.019709 sec","job_status":"done","queue_duration_s":0.769621,"scheduling_latency_s":0.769636,"redis_calls":9,"redis_duration_s":0.002198,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000731,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001052,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000415,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019709,"completed_at":"2026-09-23T19:21:26.679Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006126,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.455666] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.680Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"916ceef8342c4013d234096c","created_at":"2026-09-23T19:21:25.903Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f40636ad7db8fb75042f7545876fe18","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:25.914Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-916ceef8342c4013d234096c: start","job_status":"start","queue_duration_s":0.765662,"scheduling_latency_s":0.765676} gitlab # [ 253.463202] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.695Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"916ceef8342c4013d234096c","created_at":"2026-09-23T19:21:25.903Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f40636ad7db8fb75042f7545876fe18","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:25.914Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-916ceef8342c4013d234096c: done: 0.015133 sec","job_status":"done","queue_duration_s":0.765662,"scheduling_latency_s":0.765676,"redis_calls":11,"redis_duration_s":0.00237,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000399,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001074,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000897,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000375746,"exclusive_lock_hold_duration_s":0.002471822,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.015133,"completed_at":"2026-09-23T19:21:26.695Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001992,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.537448] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.697Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"9c26977db0cc5acd552583c0","created_at":"2026-09-23T19:21:25.923Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a94a04459b41faf1617715c7c6939860","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:25.934Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-9c26977db0cc5acd552583c0: start","job_status":"start","queue_duration_s":0.763635,"scheduling_latency_s":0.763686} gitlab # [ 253.538050] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.716Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 253.538594] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.724Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"9c26977db0cc5acd552583c0","created_at":"2026-09-23T19:21:25.923Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a94a04459b41faf1617715c7c6939860","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:25.934Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-9c26977db0cc5acd552583c0: done: 0.026653 sec","job_status":"done","queue_duration_s":0.763635,"scheduling_latency_s":0.763686,"redis_calls":9,"redis_duration_s":0.002576,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000614,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001409,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000553,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026653,"completed_at":"2026-09-23T19:21:26.724Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000658,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.581132] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.726Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"a84f30c010066995be41b0db","created_at":"2026-09-23T19:21:25.966Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f1f53af50a74e296e9ecd2e5267ccae5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:25.977Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-a84f30c010066995be41b0db: start","job_status":"start","queue_duration_s":0.748669,"scheduling_latency_s":0.748694} gitlab # [ 253.582075] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.810Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"a84f30c010066995be41b0db","created_at":"2026-09-23T19:21:25.966Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f1f53af50a74e296e9ecd2e5267ccae5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:25.977Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-a84f30c010066995be41b0db: done: 0.084611 sec","job_status":"done","queue_duration_s":0.748669,"scheduling_latency_s":0.748694,"redis_calls":12,"redis_duration_s":0.002428,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000476,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000974,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.073268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000378819,"exclusive_lock_hold_duration_s":0.070669876,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.084611,"completed_at":"2026-09-23T19:21:26.810Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006733,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null} gitlab # [ 253.632178] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.811Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b8116553cd2ebdbb221a0176","created_at":"2026-09-23T19:21:26.042Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6505df782bae077b221eb6ea96c34006","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.050Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b8116553cd2ebdbb221a0176: start","job_status":"start","queue_duration_s":0.761297,"scheduling_latency_s":0.761313} gitlab # [ 253.632861] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.876Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b8116553cd2ebdbb221a0176","created_at":"2026-09-23T19:21:26.042Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6505df782bae077b221eb6ea96c34006","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.050Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b8116553cd2ebdbb221a0176: done: 0.065064 sec","job_status":"done","queue_duration_s":0.761297,"scheduling_latency_s":0.761313,"redis_calls":11,"redis_duration_s":0.001563,"redis_read_bytes":9,"redis_write_bytes":1386,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000603,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.051,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065064,"completed_at":"2026-09-23T19:21:26.876Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.050491,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.672450] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.877Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"507ce81160d250e6de5311af","created_at":"2026-09-23T19:21:26.062Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d80265a18a2c6a6f47622c661fc91ddc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-23T19:21:26.070Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-507ce81160d250e6de5311af: start","job_status":"start","queue_duration_s":0.807657,"scheduling_latency_s":0.807676} gitlab # [ 253.673081] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.884Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 253.673546] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.892Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"507ce81160d250e6de5311af","created_at":"2026-09-23T19:21:26.062Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d80265a18a2c6a6f47622c661fc91ddc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-23T19:21:26.070Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-507ce81160d250e6de5311af: done: 0.015007 sec","job_status":"done","queue_duration_s":0.807657,"scheduling_latency_s":0.807676,"redis_calls":9,"redis_duration_s":0.001107,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000617,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00023,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015007,"completed_at":"2026-09-23T19:21:26.892Z","load_balancing_strategy":"primary","db_duration_s":0.000227,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.712471] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.893Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"1f6f713f0cc5b76ca3f7f5d1","created_at":"2026-09-23T19:21:26.081Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"560364c04593962460766838808bbe43","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-23T19:21:26.088Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-1f6f713f0cc5b76ca3f7f5d1: start","job_status":"start","queue_duration_s":0.804861,"scheduling_latency_s":0.804875} gitlab # [ 253.717735] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.911Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"1f6f713f0cc5b76ca3f7f5d1","created_at":"2026-09-23T19:21:26.081Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"560364c04593962460766838808bbe43","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-23T19:21:26.088Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-1f6f713f0cc5b76ca3f7f5d1: done: 0.017361 sec","job_status":"done","queue_duration_s":0.804861,"scheduling_latency_s":0.804875,"redis_calls":9,"redis_duration_s":0.001152,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000236,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000239,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017361,"completed_at":"2026-09-23T19:21:26.911Z","load_balancing_strategy":"primary","db_duration_s":0.00252,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.754751] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.911Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"e98b31457bc61dcee74afbbc","created_at":"2026-09-23T19:21:26.098Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38452f75404eb81487d8432209a2ffa8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.108Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-e98b31457bc61dcee74afbbc: start","job_status":"start","queue_duration_s":0.803363,"scheduling_latency_s":0.8034} gitlab # [ 253.793262] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.928Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"e98b31457bc61dcee74afbbc","created_at":"2026-09-23T19:21:26.098Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38452f75404eb81487d8432209a2ffa8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.108Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-e98b31457bc61dcee74afbbc: done: 0.016238 sec","job_status":"done","queue_duration_s":0.803363,"scheduling_latency_s":0.8034,"redis_calls":12,"redis_duration_s":0.001554,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000234,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000748,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000572,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.004606731,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.016238,"completed_at":"2026-09-23T19:21:26.928Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001442,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 253.806511] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.929Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"5c30bb8c5ab0c2c610dd58a3","created_at":"2026-09-23T19:21:26.116Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4150fd269951c671d7cf590ed1d4901","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.128Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-5c30bb8c5ab0c2c610dd58a3: start","job_status":"start","queue_duration_s":0.800155,"scheduling_latency_s":0.800169} gitlab # [ 253.819526] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.935Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 253.822581] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.939Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"5c30bb8c5ab0c2c610dd58a3","created_at":"2026-09-23T19:21:26.116Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4150fd269951c671d7cf590ed1d4901","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.128Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-5c30bb8c5ab0c2c610dd58a3: done: 0.010019 sec","job_status":"done","queue_duration_s":0.800155,"scheduling_latency_s":0.800169,"redis_calls":9,"redis_duration_s":0.00111,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000238,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00064,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000232,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010019,"completed_at":"2026-09-23T19:21:26.938Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000253,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.874240] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.939Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"cd37093113b2f3f0d15f2672","created_at":"2026-09-23T19:21:26.144Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d20886e8bae6b27eb91d15e408c2ffda","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.153Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-cd37093113b2f3f0d15f2672: start","job_status":"start","queue_duration_s":0.785909,"scheduling_latency_s":0.785923} gitlab # [ 253.874883] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.946Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses gitlab # [ 253.875725] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.949Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"cd37093113b2f3f0d15f2672","created_at":"2026-09-23T19:21:26.144Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d20886e8bae6b27eb91d15e408c2ffda","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.153Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-cd37093113b2f3f0d15f2672: done: 0.0099 sec","job_status":"done","queue_duration_s":0.785909,"scheduling_latency_s":0.785923,"redis_calls":9,"redis_duration_s":0.00106,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000237,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00059,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000233,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0099,"completed_at":"2026-09-23T19:21:26.949Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000218,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.947781] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.950Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"49a9a75ee8f9e5a0c969d9e9","created_at":"2026-09-23T19:21:26.166Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4d1bb248f33149f4f9e226b7215482e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.173Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-49a9a75ee8f9e5a0c969d9e9: start","job_status":"start","queue_duration_s":0.777213,"scheduling_latency_s":0.777226} gitlab # [ 253.948643] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.985Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"49a9a75ee8f9e5a0c969d9e9","created_at":"2026-09-23T19:21:26.166Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4d1bb248f33149f4f9e226b7215482e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.173Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-49a9a75ee8f9e5a0c969d9e9: done: 0.034616 sec","job_status":"done","queue_duration_s":0.777213,"scheduling_latency_s":0.777226,"redis_calls":13,"redis_duration_s":0.001667,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000504,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000603,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00056,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000305626,"exclusive_lock_hold_duration_s":0.022252828,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.034616,"completed_at":"2026-09-23T19:21:26.985Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006663,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 253.989136] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:26.986Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9c78f101926500f6e00e4793","created_at":"2026-09-23T19:21:26.183Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc500661a13fc9dea3cc430a6e143a69","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.194Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9c78f101926500f6e00e4793: start","job_status":"start","queue_duration_s":0.792052,"scheduling_latency_s":0.792066} gitlab # [ 253.989834] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.096Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"9c78f101926500f6e00e4793","created_at":"2026-09-23T19:21:26.183Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc500661a13fc9dea3cc430a6e143a69","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.194Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-9c78f101926500f6e00e4793: done: 0.110543 sec","job_status":"done","queue_duration_s":0.792052,"scheduling_latency_s":0.792066,"redis_calls":11,"redis_duration_s":0.001515,"redis_read_bytes":9,"redis_write_bytes":1292,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000238,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000646,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000631,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.089467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000315683,"exclusive_lock_hold_duration_s":0.098215556,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.110543,"completed_at":"2026-09-23T19:21:27.096Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024934,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.034813] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.097Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"6ee1d94a1a97aa3737c7460d","created_at":"2026-09-23T19:21:26.231Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ba599945b34c1e31944af0f97fb1514","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-23T19:21:26.241Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-6ee1d94a1a97aa3737c7460d: start","job_status":"start","queue_duration_s":0.855909,"scheduling_latency_s":0.855924} gitlab # [ 254.035570] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.104Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 254.036081] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.108Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"6ee1d94a1a97aa3737c7460d","created_at":"2026-09-23T19:21:26.231Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ba599945b34c1e31944af0f97fb1514","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-23T19:21:26.241Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-6ee1d94a1a97aa3737c7460d: done: 0.010842 sec","job_status":"done","queue_duration_s":0.855909,"scheduling_latency_s":0.855924,"redis_calls":9,"redis_duration_s":0.001187,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000301,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000654,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000232,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010842,"completed_at":"2026-09-23T19:21:27.108Z","load_balancing_strategy":"primary","db_duration_s":0.000322,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.104274] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.109Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"4dbac987e6c508d4e8fc0c7a","created_at":"2026-09-23T19:21:26.252Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.261Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-4dbac987e6c508d4e8fc0c7a: start","job_status":"start","queue_duration_s":0.848183,"scheduling_latency_s":0.848197} gitlab # [ 254.105359] nix-daemon[2823]: accepted connection from pid 3612, user nixbot gitlab # [ 254.105932] nix-daemon[2823]: accepted connection from pid 3574, user nixbot gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses, in 0.20 seconds) gitlab # [ 254.106586] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.167Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"4dbac987e6c508d4e8fc0c7a","created_at":"2026-09-23T19:21:26.252Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.261Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-4dbac987e6c508d4e8fc0c7a: done: 0.058016 sec","job_status":"done","queue_duration_s":0.848183,"scheduling_latency_s":0.848197,"redis_calls":70,"redis_duration_s":0.00865,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001115,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000463,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.006555,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000517,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.058016,"completed_at":"2026-09-23T19:21:27.167Z","load_balancing_strategy":"primary","db_duration_s":0.001132,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.119866] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.168Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"a33eee4ee2310a45475685d8","created_at":"2026-09-23T19:21:26.275Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"909bb95a029dc562850a22c426938ec0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.285Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-a33eee4ee2310a45475685d8: start","job_status":"start","queue_duration_s":0.882978,"scheduling_latency_s":0.883} gitlab # [ 254.162846] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.180Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"a33eee4ee2310a45475685d8","created_at":"2026-09-23T19:21:26.275Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"909bb95a029dc562850a22c426938ec0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.285Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-a33eee4ee2310a45475685d8: done: 0.011987 sec","job_status":"done","queue_duration_s":0.882978,"scheduling_latency_s":0.883,"redis_calls":9,"redis_duration_s":0.001148,"redis_read_bytes":7,"redis_write_bytes":1192,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000259,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000623,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.011987,"completed_at":"2026-09-23T19:21:27.180Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000441,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.201133] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.181Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"afc80838a899782a2231065a","created_at":"2026-09-23T19:21:26.303Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9b9e3afe7b86b5c758d421eaf59db28","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.312Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-afc80838a899782a2231065a: start","job_status":"start","queue_duration_s":0.868968,"scheduling_latency_s":0.868981} gitlab # [ 254.202166] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:27 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 48 gitlab # [ 254.202819] gitaly[890]: time="2026-09-23T19:21:27.397Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEF27WD4QW0BEFT1VZDFP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:57.396" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:27.396" grpc.time_ms=0.704 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 254.203433] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.197Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"afc80838a899782a2231065a","created_at":"2026-09-23T19:21:26.303Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9b9e3afe7b86b5c758d421eaf59db28","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.312Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-afc80838a899782a2231065a: done: 0.015956 sec","job_status":"done","queue_duration_s":0.868968,"scheduling_latency_s":0.868981,"redis_calls":9,"redis_duration_s":0.0011510000000000001,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000236,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000669,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015956,"completed_at":"2026-09-23T19:21:27.197Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004515,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.265726] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.198Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"3383804d0d6f88cd10da072c","created_at":"2026-09-23T19:21:26.320Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a4861be97ffc1a13551823981ad5017","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.327Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-3383804d0d6f88cd10da072c: start","job_status":"start","queue_duration_s":0.870833,"scheduling_latency_s":0.870848} gitlab # [ 254.266587] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.212Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"3383804d0d6f88cd10da072c","created_at":"2026-09-23T19:21:26.320Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a4861be97ffc1a13551823981ad5017","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.327Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-3383804d0d6f88cd10da072c: done: 0.014464 sec","job_status":"done","queue_duration_s":0.870833,"scheduling_latency_s":0.870848,"redis_calls":11,"redis_duration_s":0.001457,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000239,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000615,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000603,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000300597,"exclusive_lock_hold_duration_s":0.003892394,"duration_s":0.014464,"completed_at":"2026-09-23T19:21:27.212Z","load_balancing_strategy":"primary","db_duration_s":0.001914,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.308301] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.213Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2176c74eeefd8793f9d129e7","created_at":"2026-09-23T19:21:26.339Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f60832b425d17d2b40be03ed8ed29f8d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.347Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2176c74eeefd8793f9d129e7: start","job_status":"start","queue_duration_s":0.866492,"scheduling_latency_s":0.866505} gitlab # [ 254.308961] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.256Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"2176c74eeefd8793f9d129e7","created_at":"2026-09-23T19:21:26.339Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f60832b425d17d2b40be03ed8ed29f8d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.347Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-2176c74eeefd8793f9d129e7: done: 0.043363 sec","job_status":"done","queue_duration_s":0.866492,"scheduling_latency_s":0.866505,"redis_calls":11,"redis_duration_s":0.001588,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000657,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000673,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000329931,"exclusive_lock_hold_duration_s":0.028789413,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.043363,"completed_at":"2026-09-23T19:21:27.256Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02644,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.354689] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.258Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a980cfea81cbee5054a10ce5","created_at":"2026-09-23T19:21:26.355Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4080d311d97beb71f0f4500ff7859cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.361Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a980cfea81cbee5054a10ce5: start","job_status":"start","queue_duration_s":0.896462,"scheduling_latency_s":0.896477} gitlab # [ 254.355684] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.277Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a980cfea81cbee5054a10ce5","created_at":"2026-09-23T19:21:26.355Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-97655960d0024b1c","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4080d311d97beb71f0f4500ff7859cf8","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.361Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a980cfea81cbee5054a10ce5: done: 0.019437 sec","job_status":"done","queue_duration_s":0.896462,"scheduling_latency_s":0.896477,"redis_calls":9,"redis_duration_s":0.001375,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000278,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000851,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000246,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008685,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.019437,"completed_at":"2026-09-23T19:21:27.277Z","load_balancing_strategy":"primary","db_duration_s":0.006466,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.400753] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.278Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ea315ccb859857823b597373","created_at":"2026-09-23T19:21:26.642Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-2ea215863d2a44ed","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11d7645faaabc66c4f342bc77c1ad9ac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.653Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ea315ccb859857823b597373: start","job_status":"start","queue_duration_s":0.62492,"scheduling_latency_s":0.624935} gitlab # [ 254.401528] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.290Z","message":"No existing merge request to be cleaned up."} gitlab # [ 254.402058] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.298Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ea315ccb859857823b597373","created_at":"2026-09-23T19:21:26.642Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-2ea215863d2a44ed","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11d7645faaabc66c4f342bc77c1ad9ac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.653Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ea315ccb859857823b597373: done: 0.020075 sec","job_status":"done","queue_duration_s":0.62492,"scheduling_latency_s":0.624935,"redis_calls":12,"redis_duration_s":0.001639,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000291,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000843,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020075,"completed_at":"2026-09-23T19:21:27.298Z","load_balancing_strategy":"primary","db_duration_s":0.004856,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.468166] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.301Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"27da745fd7ba2ebb1dee0e5b","created_at":"2026-09-23T19:21:26.642Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-2ea215863d2a44ed","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11d7645faaabc66c4f342bc77c1ad9ac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.653Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-27da745fd7ba2ebb1dee0e5b: start","job_status":"start","queue_duration_s":0.64775,"scheduling_latency_s":0.647765} gitlab # [ 254.468805] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.333Z","message":"No existing merge request to be cleaned up."} gitlab # [ 254.469522] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.339Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"27da745fd7ba2ebb1dee0e5b","created_at":"2026-09-23T19:21:26.642Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-2ea215863d2a44ed","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11d7645faaabc66c4f342bc77c1ad9ac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.653Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-27da745fd7ba2ebb1dee0e5b: done: 0.038788 sec","job_status":"done","queue_duration_s":0.64775,"scheduling_latency_s":0.647765,"redis_calls":10,"redis_duration_s":0.001418,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038788,"completed_at":"2026-09-23T19:21:27.339Z","load_balancing_strategy":"primary","db_duration_s":0.051361,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.478633] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.340Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7507540acdfda7f01e290ada","created_at":"2026-09-23T19:21:26.642Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-2ea215863d2a44ed","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11d7645faaabc66c4f342bc77c1ad9ac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.653Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7507540acdfda7f01e290ada: start","job_status":"start","queue_duration_s":0.68754,"scheduling_latency_s":0.687561} gitlab # [ 254.518873] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.347Z","message":"No existing merge request to be cleaned up."} gitlab # [ 254.519640] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.351Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7507540acdfda7f01e290ada","created_at":"2026-09-23T19:21:26.642Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-2ea215863d2a44ed","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11d7645faaabc66c4f342bc77c1ad9ac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.653Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7507540acdfda7f01e290ada: done: 0.010912 sec","job_status":"done","queue_duration_s":0.68754,"scheduling_latency_s":0.687561,"redis_calls":10,"redis_duration_s":0.00131,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000776,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010912,"completed_at":"2026-09-23T19:21:27.351Z","load_balancing_strategy":"primary","db_duration_s":0.000987,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.563094] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.352Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4cb9ca9725aabaf45c3738d8","created_at":"2026-09-23T19:21:26.642Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-2ea215863d2a44ed","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11d7645faaabc66c4f342bc77c1ad9ac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.653Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4cb9ca9725aabaf45c3738d8: start","job_status":"start","queue_duration_s":0.699261,"scheduling_latency_s":0.699274} gitlab # [ 254.563734] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.361Z","message":"No existing merge request to be cleaned up."} gitlab # [ 254.564225] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.369Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4cb9ca9725aabaf45c3738d8","created_at":"2026-09-23T19:21:26.642Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-2ea215863d2a44ed","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"11d7645faaabc66c4f342bc77c1ad9ac","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:26.653Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4cb9ca9725aabaf45c3738d8: done: 0.017317 sec","job_status":"done","queue_duration_s":0.699261,"scheduling_latency_s":0.699274,"redis_calls":10,"redis_duration_s":0.0027300000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000648,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002082,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017317,"completed_at":"2026-09-23T19:21:27.369Z","load_balancing_strategy":"primary","db_duration_s":0.00458,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.572199] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da37dbb797caa8fb16cdcc2a","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da37dbb797caa8fb16cdcc2a: start","job_status":"start","queue_duration_s":0.21058,"scheduling_latency_s":0.210624} gitlab # [ 254.583442] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da37dbb797caa8fb16cdcc2a","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da37dbb797caa8fb16cdcc2a: done: 0.01328 sec","job_status":"done","queue_duration_s":0.21058,"scheduling_latency_s":0.210624,"redis_calls":15,"redis_duration_s":0.001821,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00106,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01328,"completed_at":"2026-09-23T19:21:27.384Z","load_balancing_strategy":"primary","db_duration_s":0.000294,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.620159] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"020be4198f28250c746a8439","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-020be4198f28250c746a8439: start","job_status":"start","queue_duration_s":0.224732,"scheduling_latency_s":0.224746} gitlab # [ 254.659872] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.394Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"020be4198f28250c746a8439","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-020be4198f28250c746a8439: done: 0.008953 sec","job_status":"done","queue_duration_s":0.224732,"scheduling_latency_s":0.224746,"redis_calls":13,"redis_duration_s":0.001496,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008953,"completed_at":"2026-09-23T19:21:27.394Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.665783] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58df898b18fc5a411e6141cc","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58df898b18fc5a411e6141cc: start","job_status":"start","queue_duration_s":0.234654,"scheduling_latency_s":0.234668} gitlab # [ 254.677565] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58df898b18fc5a411e6141cc","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58df898b18fc5a411e6141cc: done: 0.012214 sec","job_status":"done","queue_duration_s":0.234654,"scheduling_latency_s":0.234668,"redis_calls":13,"redis_duration_s":0.002148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000798,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00135,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012214,"completed_at":"2026-09-23T19:21:27.407Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.725604] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"584a98637734ebef8f3ce07a","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-584a98637734ebef8f3ce07a: start","job_status":"start","queue_duration_s":0.247903,"scheduling_latency_s":0.247921} gitlab # [ 254.726228] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"584a98637734ebef8f3ce07a","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-584a98637734ebef8f3ce07a: done: 0.009367 sec","job_status":"done","queue_duration_s":0.247903,"scheduling_latency_s":0.247921,"redis_calls":13,"redis_duration_s":0.0016120000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006073,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009367,"completed_at":"2026-09-23T19:21:27.417Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.758579] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d22ef25e093a2df2d7bec776","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d22ef25e093a2df2d7bec776: start","job_status":"start","queue_duration_s":0.258171,"scheduling_latency_s":0.258185} gitlab # [ 254.769606] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d22ef25e093a2df2d7bec776","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d22ef25e093a2df2d7bec776: done: 0.008765 sec","job_status":"done","queue_duration_s":0.258171,"scheduling_latency_s":0.258185,"redis_calls":13,"redis_duration_s":0.0014449999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000459,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008765,"completed_at":"2026-09-23T19:21:27.427Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.819798] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a02fdc3e0bd72183c2ef0c9a","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a02fdc3e0bd72183c2ef0c9a: start","job_status":"start","queue_duration_s":0.267782,"scheduling_latency_s":0.267796} gitlab # [ 254.820664] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.483Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a02fdc3e0bd72183c2ef0c9a","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a02fdc3e0bd72183c2ef0c9a: done: 0.055149 sec","job_status":"done","queue_duration_s":0.267782,"scheduling_latency_s":0.267796,"redis_calls":13,"redis_duration_s":0.001671,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001153,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.055149,"completed_at":"2026-09-23T19:21:27.483Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.879780] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3def18d71ad883b1f6e124c","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3def18d71ad883b1f6e124c: start","job_status":"start","queue_duration_s":0.323886,"scheduling_latency_s":0.323904} gitlab # [ 254.880650] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.492Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e3def18d71ad883b1f6e124c","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e3def18d71ad883b1f6e124c: done: 0.008687 sec","job_status":"done","queue_duration_s":0.323886,"scheduling_latency_s":0.323904,"redis_calls":13,"redis_duration_s":0.0014420000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000491,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008687,"completed_at":"2026-09-23T19:21:27.492Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.920728] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3434fb190e64169340bcf09","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3434fb190e64169340bcf09: start","job_status":"start","queue_duration_s":0.33336,"scheduling_latency_s":0.333373} gitlab # [ 254.921487] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3434fb190e64169340bcf09","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3434fb190e64169340bcf09: done: 0.013153 sec","job_status":"done","queue_duration_s":0.33336,"scheduling_latency_s":0.333373,"redis_calls":13,"redis_duration_s":0.0016460000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013153,"completed_at":"2026-09-23T19:21:27.506Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.939622] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d4ad6b1b22dcc7b9eee6a71","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d4ad6b1b22dcc7b9eee6a71: start","job_status":"start","queue_duration_s":0.347533,"scheduling_latency_s":0.347548} gitlab # [ 254.979141] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.517Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d4ad6b1b22dcc7b9eee6a71","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d4ad6b1b22dcc7b9eee6a71: done: 0.009484 sec","job_status":"done","queue_duration_s":0.347533,"scheduling_latency_s":0.347548,"redis_calls":13,"redis_duration_s":0.001403,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009484,"completed_at":"2026-09-23T19:21:27.517Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 254.984593] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37964d585485d9b6b49936f2","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37964d585485d9b6b49936f2: start","job_status":"start","queue_duration_s":0.357798,"scheduling_latency_s":0.357812} gitlab # [ 255.022737] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.527Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37964d585485d9b6b49936f2","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37964d585485d9b6b49936f2: done: 0.009294 sec","job_status":"done","queue_duration_s":0.357798,"scheduling_latency_s":0.357812,"redis_calls":13,"redis_duration_s":0.001614,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009294,"completed_at":"2026-09-23T19:21:27.527Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.028935] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.528Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4efac38a684c7c3b782890d","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4efac38a684c7c3b782890d: start","job_status":"start","queue_duration_s":0.368481,"scheduling_latency_s":0.368495} gitlab # [ 255.068814] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4efac38a684c7c3b782890d","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4efac38a684c7c3b782890d: done: 0.009083 sec","job_status":"done","queue_duration_s":0.368481,"scheduling_latency_s":0.368495,"redis_calls":13,"redis_duration_s":0.001389,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000451,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006238,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009083,"completed_at":"2026-09-23T19:21:27.537Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.073947] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39b45a3a09e99257edb4e668","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39b45a3a09e99257edb4e668: start","job_status":"start","queue_duration_s":0.37843,"scheduling_latency_s":0.378444} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses gitlab # [ 255.084731] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.547Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"39b45a3a09e99257edb4e668","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-39b45a3a09e99257edb4e668: done: 0.009036 sec","job_status":"done","queue_duration_s":0.37843,"scheduling_latency_s":0.378444,"redis_calls":13,"redis_duration_s":0.0014600000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000471,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009036,"completed_at":"2026-09-23T19:21:27.547Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.129141] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12f796c53539a8286089f2f8","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12f796c53539a8286089f2f8: start","job_status":"start","queue_duration_s":0.38828,"scheduling_latency_s":0.388293} gitlab # [ 255.129843] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12f796c53539a8286089f2f8","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12f796c53539a8286089f2f8: done: 0.009541 sec","job_status":"done","queue_duration_s":0.38828,"scheduling_latency_s":0.388293,"redis_calls":13,"redis_duration_s":0.00144,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000461,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009541,"completed_at":"2026-09-23T19:21:27.558Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.191912] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.559Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98fd0b9464093a8b7ea2e3a2","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98fd0b9464093a8b7ea2e3a2: start","job_status":"start","queue_duration_s":0.39863,"scheduling_latency_s":0.398643} gitlab # [ 255.192688] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98fd0b9464093a8b7ea2e3a2","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98fd0b9464093a8b7ea2e3a2: done: 0.009067 sec","job_status":"done","queue_duration_s":0.39863,"scheduling_latency_s":0.398643,"redis_calls":13,"redis_duration_s":0.001414,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000451,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009067,"completed_at":"2026-09-23T19:21:27.568Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.208712] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f912bbf797507d941356cdb8","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f912bbf797507d941356cdb8: start","job_status":"start","queue_duration_s":0.40847,"scheduling_latency_s":0.408484} gitlab # [ 255.219671] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.578Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f912bbf797507d941356cdb8","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f912bbf797507d941356cdb8: done: 0.009258 sec","job_status":"done","queue_duration_s":0.40847,"scheduling_latency_s":0.408484,"redis_calls":13,"redis_duration_s":0.001653,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009258,"completed_at":"2026-09-23T19:21:27.578Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.266181] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.579Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d975c953600be820bc3ddeea","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d975c953600be820bc3ddeea: start","job_status":"start","queue_duration_s":0.418993,"scheduling_latency_s":0.419006} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses, in 0.16 seconds) gitlab # [ 255.266891] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d975c953600be820bc3ddeea","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d975c953600be820bc3ddeea: done: 0.012388 sec","job_status":"done","queue_duration_s":0.418993,"scheduling_latency_s":0.419006,"redis_calls":13,"redis_duration_s":0.002358,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001818,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.012388,"completed_at":"2026-09-23T19:21:27.591Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.307567] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d2c19697b64e1788a206ac1","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d2c19697b64e1788a206ac1: start","job_status":"start","queue_duration_s":0.432365,"scheduling_latency_s":0.432379} gitlab # [ 255.344881] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d2c19697b64e1788a206ac1","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d2c19697b64e1788a206ac1: done: 0.009231 sec","job_status":"done","queue_duration_s":0.432365,"scheduling_latency_s":0.432379,"redis_calls":13,"redis_duration_s":0.0014500000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000476,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000974,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009231,"completed_at":"2026-09-23T19:21:27.601Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.346044] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.603Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44bd91a4b074d2d8aed06426","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44bd91a4b074d2d8aed06426: start","job_status":"start","queue_duration_s":0.442923,"scheduling_latency_s":0.442936} gitlab # [ 255.385968] gitaly[890]: time="2026-09-23T19:21:28.563Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEG3WDHMZQCD5W1FT42FD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:58.062" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:28.562" grpc.time_ms=0.471 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 255.386895] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:28 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 133 gitlab # [ 255.387474] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44bd91a4b074d2d8aed06426","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44bd91a4b074d2d8aed06426: done: 0.008891 sec","job_status":"done","queue_duration_s":0.442923,"scheduling_latency_s":0.442936,"redis_calls":13,"redis_duration_s":0.001471,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000479,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008891,"completed_at":"2026-09-23T19:21:27.612Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.430727] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cdfc369fc7d455179dbe8aa1","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cdfc369fc7d455179dbe8aa1: start","job_status":"start","queue_duration_s":0.452586,"scheduling_latency_s":0.452616} gitlab # [ 255.431649] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cdfc369fc7d455179dbe8aa1","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cdfc369fc7d455179dbe8aa1: done: 0.008896 sec","job_status":"done","queue_duration_s":0.452586,"scheduling_latency_s":0.452616,"redis_calls":13,"redis_duration_s":0.001426,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000474,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008896,"completed_at":"2026-09-23T19:21:27.621Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.471750] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.623Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53e788b869266890a7f12e76","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53e788b869266890a7f12e76: start","job_status":"start","queue_duration_s":0.462831,"scheduling_latency_s":0.462843} gitlab # [ 255.472544] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.633Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53e788b869266890a7f12e76","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53e788b869266890a7f12e76: done: 0.010374 sec","job_status":"done","queue_duration_s":0.462831,"scheduling_latency_s":0.462843,"redis_calls":13,"redis_duration_s":0.0021200000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000657,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001463,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010374,"completed_at":"2026-09-23T19:21:27.633Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.491258] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6239ef36ff23d92d1c30b4f","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6239ef36ff23d92d1c30b4f: start","job_status":"start","queue_duration_s":0.474331,"scheduling_latency_s":0.47435} gitlab # [ 255.502652] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.645Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6239ef36ff23d92d1c30b4f","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6239ef36ff23d92d1c30b4f: done: 0.010886 sec","job_status":"done","queue_duration_s":0.474331,"scheduling_latency_s":0.47435,"redis_calls":13,"redis_duration_s":0.001438,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00046,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.010886,"completed_at":"2026-09-23T19:21:27.645Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.545039] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.646Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0f30b32d6a54971f764192d","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0f30b32d6a54971f764192d: start","job_status":"start","queue_duration_s":0.485998,"scheduling_latency_s":0.486011} gitlab # [ 255.547730] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.655Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0f30b32d6a54971f764192d","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0f30b32d6a54971f764192d: done: 0.00918 sec","job_status":"done","queue_duration_s":0.485998,"scheduling_latency_s":0.486011,"redis_calls":13,"redis_duration_s":0.001427,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000471,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00918,"completed_at":"2026-09-23T19:21:27.655Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.607129] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64db166aaeea50e46682439d","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64db166aaeea50e46682439d: start","job_status":"start","queue_duration_s":0.496486,"scheduling_latency_s":0.496499} gitlab # [ 255.607612] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64db166aaeea50e46682439d","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64db166aaeea50e46682439d: done: 0.00931 sec","job_status":"done","queue_duration_s":0.496486,"scheduling_latency_s":0.496499,"redis_calls":13,"redis_duration_s":0.001416,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00045,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00931,"completed_at":"2026-09-23T19:21:27.666Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.626841] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f664df88cef724630b86987","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f664df88cef724630b86987: start","job_status":"start","queue_duration_s":0.506567,"scheduling_latency_s":0.506581} gitlab # [ 255.666053] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f664df88cef724630b86987","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f664df88cef724630b86987: done: 0.009224 sec","job_status":"done","queue_duration_s":0.506567,"scheduling_latency_s":0.506581,"redis_calls":13,"redis_duration_s":0.001627,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00047,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009224,"completed_at":"2026-09-23T19:21:27.676Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.671959] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"666f520afff810ab776ee7bf","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-666f520afff810ab776ee7bf: start","job_status":"start","queue_duration_s":0.5172,"scheduling_latency_s":0.517215} gitlab # [ 255.683123] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"666f520afff810ab776ee7bf","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-666f520afff810ab776ee7bf: done: 0.011231 sec","job_status":"done","queue_duration_s":0.5172,"scheduling_latency_s":0.517215,"redis_calls":13,"redis_duration_s":0.001752,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001251,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011231,"completed_at":"2026-09-23T19:21:27.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.751109] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86887f8a86a15535c7113a42","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86887f8a86a15535c7113a42: start","job_status":"start","queue_duration_s":0.529499,"scheduling_latency_s":0.529517} gitlab # [ 255.751650] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86887f8a86a15535c7113a42","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86887f8a86a15535c7113a42: done: 0.011061 sec","job_status":"done","queue_duration_s":0.529499,"scheduling_latency_s":0.529517,"redis_calls":13,"redis_duration_s":0.001665,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.011061,"completed_at":"2026-09-23T19:21:27.700Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.763231] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.701Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"713f83f6f29d622c6637f87c","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-713f83f6f29d622c6637f87c: start","job_status":"start","queue_duration_s":0.541533,"scheduling_latency_s":0.541549} gitlab # [ 255.774632] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"713f83f6f29d622c6637f87c","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-713f83f6f29d622c6637f87c: done: 0.009652 sec","job_status":"done","queue_duration_s":0.541533,"scheduling_latency_s":0.541549,"redis_calls":13,"redis_duration_s":0.001542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009652,"completed_at":"2026-09-23T19:21:27.711Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.839048] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bf7dc27036238864ab7b4ed","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bf7dc27036238864ab7b4ed: start","job_status":"start","queue_duration_s":0.552626,"scheduling_latency_s":0.55264} gitlab # [ 255.839601] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0bf7dc27036238864ab7b4ed","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0bf7dc27036238864ab7b4ed: done: 0.00947 sec","job_status":"done","queue_duration_s":0.552626,"scheduling_latency_s":0.55264,"redis_calls":13,"redis_duration_s":0.001515,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00947,"completed_at":"2026-09-23T19:21:27.722Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.879123] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3ef2eb22501c72502536d60","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3ef2eb22501c72502536d60: start","job_status":"start","queue_duration_s":0.562911,"scheduling_latency_s":0.562923} gitlab # [ 255.879860] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3ef2eb22501c72502536d60","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3ef2eb22501c72502536d60: done: 0.014445 sec","job_status":"done","queue_duration_s":0.562911,"scheduling_latency_s":0.562923,"redis_calls":13,"redis_duration_s":0.0016769999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001143,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014445,"completed_at":"2026-09-23T19:21:27.737Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.926044] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.739Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da2ea696b5310454ea4e44fd","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da2ea696b5310454ea4e44fd: start","job_status":"start","queue_duration_s":0.578655,"scheduling_latency_s":0.578669} gitlab # [ 255.926769] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da2ea696b5310454ea4e44fd","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da2ea696b5310454ea4e44fd: done: 0.009533 sec","job_status":"done","queue_duration_s":0.578655,"scheduling_latency_s":0.578669,"redis_calls":13,"redis_duration_s":0.001567,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001056,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009533,"completed_at":"2026-09-23T19:21:27.748Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 255.977628] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47109db8d13728a30b8cedf9","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47109db8d13728a30b8cedf9: start","job_status":"start","queue_duration_s":0.589057,"scheduling_latency_s":0.589071} gitlab # [ 255.978239] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.758Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47109db8d13728a30b8cedf9","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47109db8d13728a30b8cedf9: done: 0.009127 sec","job_status":"done","queue_duration_s":0.589057,"scheduling_latency_s":0.589071,"redis_calls":13,"redis_duration_s":0.001549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009127,"completed_at":"2026-09-23T19:21:27.758Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.019484] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ceaf18cbf8e870270775866","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ceaf18cbf8e870270775866: start","job_status":"start","queue_duration_s":0.599012,"scheduling_latency_s":0.599025} gitlab # [ 256.020192] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ceaf18cbf8e870270775866","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ceaf18cbf8e870270775866: done: 0.009358 sec","job_status":"done","queue_duration_s":0.599012,"scheduling_latency_s":0.599025,"redis_calls":13,"redis_duration_s":0.0015409999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009358,"completed_at":"2026-09-23T19:21:27.768Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.063702] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"922e7f70b7672477a2b13448","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-922e7f70b7672477a2b13448: start","job_status":"start","queue_duration_s":0.609198,"scheduling_latency_s":0.609212} gitlab # [ 256.064422] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"922e7f70b7672477a2b13448","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-922e7f70b7672477a2b13448: done: 0.009316 sec","job_status":"done","queue_duration_s":0.609198,"scheduling_latency_s":0.609212,"redis_calls":13,"redis_duration_s":0.0017490000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009316,"completed_at":"2026-09-23T19:21:27.778Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.101851] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64065c18a1547a705cd46edb","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64065c18a1547a705cd46edb: start","job_status":"start","queue_duration_s":0.619356,"scheduling_latency_s":0.619371} gitlab # [ 256.102725] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64065c18a1547a705cd46edb","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64065c18a1547a705cd46edb: done: 0.009117 sec","job_status":"done","queue_duration_s":0.619356,"scheduling_latency_s":0.619371,"redis_calls":13,"redis_duration_s":0.001508,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000493,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009117,"completed_at":"2026-09-23T19:21:27.788Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.145481] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e6036999afea3b3dcbd6032","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e6036999afea3b3dcbd6032: start","job_status":"start","queue_duration_s":0.629293,"scheduling_latency_s":0.629307} gitlab # [ 256.146146] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e6036999afea3b3dcbd6032","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e6036999afea3b3dcbd6032: done: 0.009115 sec","job_status":"done","queue_duration_s":0.629293,"scheduling_latency_s":0.629307,"redis_calls":13,"redis_duration_s":0.001633,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001134,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009115,"completed_at":"2026-09-23T19:21:27.798Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.168943] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1121063de260002033591f93","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1121063de260002033591f93: start","job_status":"start","queue_duration_s":0.639231,"scheduling_latency_s":0.639244} gitlab # [ 256.211211] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.808Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1121063de260002033591f93","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1121063de260002033591f93: done: 0.008818 sec","job_status":"done","queue_duration_s":0.639231,"scheduling_latency_s":0.639244,"redis_calls":13,"redis_duration_s":0.001545,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008818,"completed_at":"2026-09-23T19:21:27.808Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.214482] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e83057938a390f5ab5884d52","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e83057938a390f5ab5884d52: start","job_status":"start","queue_duration_s":0.648885,"scheduling_latency_s":0.648898} gitlab # [ 256.262092] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e83057938a390f5ab5884d52","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e83057938a390f5ab5884d52: done: 0.00884 sec","job_status":"done","queue_duration_s":0.648885,"scheduling_latency_s":0.648898,"redis_calls":13,"redis_duration_s":0.001546,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00884,"completed_at":"2026-09-23T19:21:27.818Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.263150] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fc2e55e341f87a9a3887e4a","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fc2e55e341f87a9a3887e4a: start","job_status":"start","queue_duration_s":0.658713,"scheduling_latency_s":0.658726} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses gitlab # [ 256.270943] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fc2e55e341f87a9a3887e4a","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fc2e55e341f87a9a3887e4a: done: 0.008876 sec","job_status":"done","queue_duration_s":0.658713,"scheduling_latency_s":0.658726,"redis_calls":13,"redis_duration_s":0.00166,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008876,"completed_at":"2026-09-23T19:21:27.827Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.318110] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b684c8374ce5e3496ca8477f","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b684c8374ce5e3496ca8477f: start","job_status":"start","queue_duration_s":0.66846,"scheduling_latency_s":0.668472} gitlab # [ 256.318784] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b684c8374ce5e3496ca8477f","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b684c8374ce5e3496ca8477f: done: 0.009075 sec","job_status":"done","queue_duration_s":0.66846,"scheduling_latency_s":0.668472,"redis_calls":13,"redis_duration_s":0.001614,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009075,"completed_at":"2026-09-23T19:21:27.837Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.374110] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.838Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77bd025b56394f06f0d95bc0","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77bd025b56394f06f0d95bc0: start","job_status":"start","queue_duration_s":0.678367,"scheduling_latency_s":0.67838} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses, in 0.11 seconds) gitlab # [ 256.374986] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77bd025b56394f06f0d95bc0","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77bd025b56394f06f0d95bc0: done: 0.009087 sec","job_status":"done","queue_duration_s":0.678367,"scheduling_latency_s":0.67838,"redis_calls":13,"redis_duration_s":0.0016719999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001171,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009087,"completed_at":"2026-09-23T19:21:27.847Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.421825] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7549dc01d5baadeb7722f4f4","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7549dc01d5baadeb7722f4f4: start","job_status":"start","queue_duration_s":0.688387,"scheduling_latency_s":0.688401} gitlab # [ 256.422915] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7549dc01d5baadeb7722f4f4","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7549dc01d5baadeb7722f4f4: done: 0.009042 sec","job_status":"done","queue_duration_s":0.688387,"scheduling_latency_s":0.688401,"redis_calls":13,"redis_duration_s":0.001549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009042,"completed_at":"2026-09-23T19:21:27.857Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.442313] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db0f5426f549ec6f5ed9f177","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db0f5426f549ec6f5ed9f177: start","job_status":"start","queue_duration_s":0.698287,"scheduling_latency_s":0.6983} gitlab # [ 256.454168] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 60 gitlab # [ 256.456332] gitaly[890]: time="2026-09-23T19:21:29.669Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEH8T6JZWC07V4DBW6DY6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:21:59.167" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:29.668" grpc.time_ms=0.69 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 256.503773] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db0f5426f549ec6f5ed9f177","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db0f5426f549ec6f5ed9f177: done: 0.009081 sec","job_status":"done","queue_duration_s":0.698287,"scheduling_latency_s":0.6983,"redis_calls":13,"redis_duration_s":0.001559,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009081,"completed_at":"2026-09-23T19:21:27.867Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.504931] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b8fd67a284de7364cba544b","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b8fd67a284de7364cba544b: start","job_status":"start","queue_duration_s":0.708245,"scheduling_latency_s":0.70826} gitlab # [ 256.544413] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b8fd67a284de7364cba544b","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b8fd67a284de7364cba544b: done: 0.009141 sec","job_status":"done","queue_duration_s":0.708245,"scheduling_latency_s":0.70826,"redis_calls":13,"redis_duration_s":0.0018009999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001236,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009141,"completed_at":"2026-09-23T19:21:27.877Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.583256] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ff1e0e8c1604de61ecc97b3","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ff1e0e8c1604de61ecc97b3: start","job_status":"start","queue_duration_s":0.743459,"scheduling_latency_s":0.74348} gitlab # [ 256.584065] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.913Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ff1e0e8c1604de61ecc97b3","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ff1e0e8c1604de61ecc97b3: done: 0.009934 sec","job_status":"done","queue_duration_s":0.743459,"scheduling_latency_s":0.74348,"redis_calls":13,"redis_duration_s":0.001562,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001063,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.009934,"completed_at":"2026-09-23T19:21:27.913Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.591593] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"632bbd2bbb9bea97502465f7","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-632bbd2bbb9bea97502465f7: start","job_status":"start","queue_duration_s":0.754212,"scheduling_latency_s":0.754225} gitlab # [ 256.636232] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"632bbd2bbb9bea97502465f7","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-632bbd2bbb9bea97502465f7: done: 0.00944 sec","job_status":"done","queue_duration_s":0.754212,"scheduling_latency_s":0.754225,"redis_calls":13,"redis_duration_s":0.001682,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001175,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.00944,"completed_at":"2026-09-23T19:21:27.923Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.636792] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0799be7da47142a8e1a0a76","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0799be7da47142a8e1a0a76: start","job_status":"start","queue_duration_s":0.76445,"scheduling_latency_s":0.764463} gitlab # [ 256.676143] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0799be7da47142a8e1a0a76","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0799be7da47142a8e1a0a76: done: 0.008755 sec","job_status":"done","queue_duration_s":0.76445,"scheduling_latency_s":0.764463,"redis_calls":13,"redis_duration_s":0.001418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00046,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008755,"completed_at":"2026-09-23T19:21:27.933Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.681686] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2117c8c760b095a644a16f5","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2117c8c760b095a644a16f5: start","job_status":"start","queue_duration_s":0.774159,"scheduling_latency_s":0.774173} gitlab # [ 256.692670] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.943Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2117c8c760b095a644a16f5","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2117c8c760b095a644a16f5: done: 0.008692 sec","job_status":"done","queue_duration_s":0.774159,"scheduling_latency_s":0.774173,"redis_calls":13,"redis_duration_s":0.001436,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000466,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008692,"completed_at":"2026-09-23T19:21:27.943Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.743088] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.944Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"538ae7158911360e0fa341dc","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-538ae7158911360e0fa341dc: start","job_status":"start","queue_duration_s":0.783641,"scheduling_latency_s":0.783654} gitlab # [ 256.743811] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"538ae7158911360e0fa341dc","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-538ae7158911360e0fa341dc: done: 0.01052 sec","job_status":"done","queue_duration_s":0.783641,"scheduling_latency_s":0.783654,"redis_calls":13,"redis_duration_s":0.001797,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000594,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01052,"completed_at":"2026-09-23T19:21:27.954Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.772715] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e48769cf029c6a179fce3651","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e48769cf029c6a179fce3651: start","job_status":"start","queue_duration_s":0.795185,"scheduling_latency_s":0.7952} gitlab # [ 256.818220] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e48769cf029c6a179fce3651","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e48769cf029c6a179fce3651: done: 0.008801 sec","job_status":"done","queue_duration_s":0.795185,"scheduling_latency_s":0.7952,"redis_calls":13,"redis_duration_s":0.001439,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000453,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00594,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008801,"completed_at":"2026-09-23T19:21:27.964Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.819224] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4780c229f391b3bfb5ddc07c","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4780c229f391b3bfb5ddc07c: start","job_status":"start","queue_duration_s":0.804797,"scheduling_latency_s":0.80481} gitlab # [ 256.860299] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:27.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4780c229f391b3bfb5ddc07c","created_at":"2026-09-23T19:21:27.120Z","trace_propagation_headers":{"sentry-trace":"0ffc7b9d742a4f6d991dce7e82a2e173-a01da38e30e0453e","baggage":"sentry-trace_id=0ffc7b9d742a4f6d991dce7e82a2e173,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"28a727fb5387265e04242b6cdbf54d40","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-23T19:21:27.160Z","job_size_bytes":2,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4780c229f391b3bfb5ddc07c: done: 0.008743 sec","job_status":"done","queue_duration_s":0.804797,"scheduling_latency_s":0.80481,"redis_calls":13,"redis_duration_s":0.001409,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000454,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.008743,"completed_at":"2026-09-23T19:21:27.973Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 256.958606] gitaly[890]: time="2026-09-23T19:21:30.231Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:22:00.229" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:30.230" grpc.time_ms=0.893 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.016614] gitaly[890]: time="2026-09-23T19:21:30.289Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=626 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:21:40.076" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:30.277" grpc.time_ms=11.966 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.096212] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.368Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"47cac7641fa4121e9c00f8fe","created_at":"2026-09-23T19:21:30.361Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b620fd59f0c14461","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.366Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-47cac7641fa4121e9c00f8fe: start","job_status":"start","queue_duration_s":0.002108,"scheduling_latency_s":0.002157} gitlab # [ 257.130231] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/263054a42f00ab5c2554e36527725bb48ab5e4c1 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 208 gitlab # [ 257.136655] nixbot[2658]: {"ts": "2026-09-23T19:21:30.402Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/263054a42f00ab5c2554e36527725bb48ab5e4c1 \"HTTP/1.1 201 Created\""} gitlab # [ 257.205732] gitaly[890]: time="2026-09-23T19:21:30.478Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEJ17S5DDY09QFECWYKR6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:22:00.076" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:30.477" grpc.time_ms=0.8 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.242247] gitaly[890]: time="2026-09-23T19:21:30.515Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:30.514" grpc.time_ms=0.964 pid=890 span.kind=server system=grpc gitlab # [ 257.258931] gitaly[890]: time="2026-09-23T19:21:30.531Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=622 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VEJ17S5DDY09QFECWYKR6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:21:40.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:30.526" grpc.time_ms=5.478 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.287640] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/263054a42f00ab5c2554e36527725bb48ab5e4c1 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 153 gitlab # [ 257.288441] nixbot[2658]: {"ts": "2026-09-23T19:21:30.561Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/263054a42f00ab5c2554e36527725bb48ab5e4c1 \"HTTP/1.1 201 Created\""} gitlab # [ 257.290895] gitaly[890]: time="2026-09-23T19:21:30.563Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:30.563" grpc.time_ms=0.26 pid=890 span.kind=server system=grpc gitlab # [ 257.319249] gitaly[890]: time="2026-09-23T19:21:30.592Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:30.591" grpc.time_ms=0.264 pid=890 span.kind=server system=grpc gitlab # [ 257.335157] gitaly[890]: time="2026-09-23T19:21:30.607Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY diskcache=7da1dd61-ba5f-4122-acd2-b3cbd1e8f52b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:22:00.488" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:30.489" pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.340759] gitaly[890]: time="2026-09-23T19:21:30.613Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEJ6D3BBAHK93Z2JSY61P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:22:00.212" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:30.613" grpc.time_ms=0.418 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.355198] gitaly[890]: time="2026-09-23T19:21:30.627Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=72 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7438 command.oublock=0 command.real_time_ms=111 command.spawn_token_fork_ms=0 command.system_time_ms=44 command.user_time_ms=28 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:22:00.488" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:30.489" grpc.time_ms=118.551 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.384395] gitaly[890]: time="2026-09-23T19:21:30.657Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M37VEJ6D3BBAHK93Z2JSY61P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:21:40.129" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:30.650" grpc.time_ms=6.956 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.403319] gitaly[890]: time="2026-09-23T19:21:30.676Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-23T19:22:00.675" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:30.675" grpc.time_ms=0.693 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses gitlab # [ 257.431099] gitaly[890]: time="2026-09-23T19:21:30.703Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:22:00.702" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:30.703" grpc.time_ms=0.469 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.433969] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/263054a42f00ab5c2554e36527725bb48ab5e4c1 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 133 gitlab # [ 257.434866] nixbot[2658]: {"ts": "2026-09-23T19:21:30.707Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/263054a42f00ab5c2554e36527725bb48ab5e4c1 \"HTTP/1.1 201 Created\""} gitlab # [ 257.459429] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.731Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"cc05ae6ee0b88de647fa119b","created_at":"2026-09-23T19:21:30.725Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"47cac7641fa4121e9c00f8fe","job_size_bytes":3,"pid":2530,"job_status":"deduplicated","message":"PipelineProcessWorker JID-cc05ae6ee0b88de647fa119b: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 257.473912] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.746Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"47cac7641fa4121e9c00f8fe","created_at":"2026-09-23T19:21:30.361Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b620fd59f0c14461","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.366Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2530,"job_status":"rescheduled","message":"PipelineProcessWorker JID-47cac7641fa4121e9c00f8fe: rescheduled"} gitlab # [ 257.491357] gitaly[890]: time="2026-09-23T19:21:30.763Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEJAR9EWEFDR7QCENN22X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:22:00.762" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:30.763" grpc.time_ms=0.538 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.503243] gitaly[890]: time="2026-09-23T19:21:30.775Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEJB6XB4JRF1Y655X6W0D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:22:00.374" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:30.775" grpc.time_ms=0.511 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.503575] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.764Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"47cac7641fa4121e9c00f8fe","created_at":"2026-09-23T19:21:30.361Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b620fd59f0c14461","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.366Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-47cac7641fa4121e9c00f8fe: done: 0.396296 sec","job_status":"done","queue_duration_s":0.002108,"scheduling_latency_s":0.002157,"gitaly_calls":3,"gitaly_duration_s":0.160746,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.04306,"redis_read_bytes":239,"redis_write_bytes":21779,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001477,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000961,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.002597,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14464,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.034283,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000285,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000447,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001533,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.06,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.011,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.015,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.132389,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.303734972,"duration_s":0.396296,"completed_at":"2026-09-23T19:21:30.764Z","load_balancing_strategy":"primary","db_duration_s":0.064442,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 257.521563] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.769Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8c7e730d195f6061db035ee2","created_at":"2026-09-23T19:21:30.482Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.483Z","job_size_bytes":23,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8c7e730d195f6061db035ee2: start","job_status":"start","queue_duration_s":0.285943,"scheduling_latency_s":0.285976} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses, in 0.17 seconds) gitlab # [ 257.536323] gitaly[890]: time="2026-09-23T19:21:30.790Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=624 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M37VEJAR9EWEFDR7QCENN22X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-23T19:21:40.254" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:30.784" grpc.time_ms=5.276 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.548483] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.814Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8c7e730d195f6061db035ee2","created_at":"2026-09-23T19:21:30.482Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.483Z","job_size_bytes":23,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8c7e730d195f6061db035ee2: done: 0.044592 sec","job_status":"done","queue_duration_s":0.285943,"scheduling_latency_s":0.285976,"redis_calls":7,"redis_duration_s":0.002574,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001284,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044592,"completed_at":"2026-09-23T19:21:30.814Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013139,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 257.596329] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses HTTP/1.1" 200 1313 "" "curl/8.21.0" 89 gitlab # [ 257.597038] gitlab-workhorse[825]: gitlab 2001:db8:1::1 - - [2026/09/23:19:21:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/263054a42f00ab5c2554e36527725bb48ab5e4c1 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 107 gitlab # [ 257.597586] nixbot[2658]: {"ts": "2026-09-23T19:21:30.821Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/263054a42f00ab5c2554e36527725bb48ab5e4c1 \"HTTP/1.1 201 Created\""} gitlab # [ 257.598071] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.824Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9019445f3c53317866f301cd","created_at":"2026-09-23T19:21:30.633Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.635Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9019445f3c53317866f301cd: start","job_status":"start","queue_duration_s":0.189061,"scheduling_latency_s":0.189092} gitlab # [ 257.639903] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.856Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9019445f3c53317866f301cd","created_at":"2026-09-23T19:21:30.633Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.635Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9019445f3c53317866f301cd: done: 0.031874 sec","job_status":"done","queue_duration_s":0.189061,"scheduling_latency_s":0.189092,"redis_calls":7,"redis_duration_s":0.00107,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000329,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031874,"completed_at":"2026-09-23T19:21:30.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013322,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.646291] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.857Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"129c3b098941ecda0a680d5c","created_at":"2026-09-23T19:21:30.642Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.645Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-129c3b098941ecda0a680d5c: start","job_status":"start","queue_duration_s":0.211835,"scheduling_latency_s":0.21185} gitlab # [ 257.660821] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.878Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"129c3b098941ecda0a680d5c","created_at":"2026-09-23T19:21:30.642Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.645Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-129c3b098941ecda0a680d5c: done: 0.020984 sec","job_status":"done","queue_duration_s":0.211835,"scheduling_latency_s":0.21185,"redis_calls":7,"redis_duration_s":0.001248,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00074,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000508,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020984,"completed_at":"2026-09-23T19:21:30.878Z","load_balancing_strategy":"primary","db_duration_s":0.004585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.705128] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.879Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"52ca23f0b11100fe90720695","created_at":"2026-09-23T19:21:30.652Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.654Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-52ca23f0b11100fe90720695: start","job_status":"start","queue_duration_s":0.22461,"scheduling_latency_s":0.224629} gitlab # [ 257.743781] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.900Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"52ca23f0b11100fe90720695","created_at":"2026-09-23T19:21:30.652Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.654Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-52ca23f0b11100fe90720695: done: 0.021565 sec","job_status":"done","queue_duration_s":0.22461,"scheduling_latency_s":0.224629,"redis_calls":10,"redis_duration_s":0.001389,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000726,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000379,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021565,"completed_at":"2026-09-23T19:21:30.900Z","load_balancing_strategy":"primary","db_duration_s":0.003305,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.783886] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.901Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"89fcc979fe2480535414b0a3","created_at":"2026-09-23T19:21:30.655Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.656Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-89fcc979fe2480535414b0a3: start","job_status":"start","queue_duration_s":0.245585,"scheduling_latency_s":0.245607} gitlab # [ 257.784625] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.925Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"89fcc979fe2480535414b0a3","created_at":"2026-09-23T19:21:30.655Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.656Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-89fcc979fe2480535414b0a3: done: 0.023582 sec","job_status":"done","queue_duration_s":0.245585,"scheduling_latency_s":0.245607,"redis_calls":7,"redis_duration_s":0.003188,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002276,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023582,"completed_at":"2026-09-23T19:21:30.925Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006959,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.824994] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.928Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"5bb9fdef87efeb183cee0b01","created_at":"2026-09-23T19:21:30.656Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.658Z","job_size_bytes":19,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-5bb9fdef87efeb183cee0b01: start","job_status":"start","queue_duration_s":0.269486,"scheduling_latency_s":0.2695} gitlab # [ 257.825656] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.947Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"5bb9fdef87efeb183cee0b01","created_at":"2026-09-23T19:21:30.656Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.658Z","job_size_bytes":19,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-5bb9fdef87efeb183cee0b01: done: 0.019559 sec","job_status":"done","queue_duration_s":0.269486,"scheduling_latency_s":0.2695,"redis_calls":8,"redis_duration_s":0.00213,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001776,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000354,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019559,"completed_at":"2026-09-23T19:21:30.947Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003894,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.866106] gitaly[890]: time="2026-09-23T19:21:31.095Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:31.095" grpc.time_ms=0.409 pid=890 span.kind=server system=grpc gitlab # [ 257.866727] gitaly[890]: time="2026-09-23T19:21:31.116Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3892 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:31.115" grpc.time_ms=0.203 pid=890 span.kind=server system=grpc gitlab # [ 257.907348] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.948Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"4e21a14d159a72aa51955128","created_at":"2026-09-23T19:21:30.659Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.660Z","job_size_bytes":66,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-4e21a14d159a72aa51955128: start","job_status":"start","queue_duration_s":0.288028,"scheduling_latency_s":0.288043} gitlab # [ 257.908040] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.962Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"4e21a14d159a72aa51955128","created_at":"2026-09-23T19:21:30.659Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.660Z","job_size_bytes":66,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-4e21a14d159a72aa51955128: done: 0.014267 sec","job_status":"done","queue_duration_s":0.288028,"scheduling_latency_s":0.288043,"redis_calls":7,"redis_duration_s":0.001098,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000239,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014267,"completed_at":"2026-09-23T19:21:30.962Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002293,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.947301] gitaly[890]: time="2026-09-23T19:21:31.133Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-23T19:21:31.133" grpc.time_ms=0.625 pid=890 span.kind=server system=grpc gitlab # [ 257.947938] gitaly[890]: time="2026-09-23T19:21:31.137Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:22:01.075" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:31.076" operations="[repack_refs]" pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 257.948618] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.963Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"316b9f3828c7fa11f5d8a71f","created_at":"2026-09-23T19:21:30.661Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.662Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-316b9f3828c7fa11f5d8a71f: start","job_status":"start","queue_duration_s":0.301629,"scheduling_latency_s":0.301643} gitlab # [ 257.957488] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.984Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"316b9f3828c7fa11f5d8a71f","created_at":"2026-09-23T19:21:30.661Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.662Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-316b9f3828c7fa11f5d8a71f: done: 0.02047 sec","job_status":"done","queue_duration_s":0.301629,"scheduling_latency_s":0.301643,"redis_calls":7,"redis_duration_s":0.000864,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000633,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000231,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02047,"completed_at":"2026-09-23T19:21:30.984Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0042,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 257.999020] gitaly[890]: time="2026-09-23T19:21:31.139Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:22:01.075" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:31.076" pid=890 remote_ip="2001:db8:1::1" repository_info="{false {6 702 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-23 19:20:24.489120383 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root gitlab # [ 258.005611] gitaly[890]: time="2026-09-23T19:21:31.143Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY diskcache=df3e4764-551d-4e0e-84a8-2c06f36b95d6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:22:01.075" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:31.076" pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.047070] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:30.985Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"a29cb5ed96767d0af6ab157d","created_at":"2026-09-23T19:21:30.665Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.666Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-a29cb5ed96767d0af6ab157d: start","job_status":"start","queue_duration_s":0.318828,"scheduling_latency_s":0.318842} gitlab # [ 258.047709] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.003Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"a29cb5ed96767d0af6ab157d","created_at":"2026-09-23T19:21:30.665Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.666Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-a29cb5ed96767d0af6ab157d: done: 0.018227 sec","job_status":"done","queue_duration_s":0.318828,"scheduling_latency_s":0.318842,"redis_calls":7,"redis_duration_s":0.001001,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00077,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000231,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018227,"completed_at":"2026-09-23T19:21:31.003Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004104,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.087866] gitaly[890]: time="2026-09-23T19:21:31.144Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=67 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7793 command.oublock=0 command.real_time_ms=62 command.spawn_token_fork_ms=0 command.system_time_ms=35 command.user_time_ms=32 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:22:01.075" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:31.076" grpc.time_ms=67.745 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.088748] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.004Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"ab54cab88bf8a01db75a64a9","created_at":"2026-09-23T19:21:30.666Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.667Z","job_size_bytes":6,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-ab54cab88bf8a01db75a64a9: start","job_status":"start","queue_duration_s":0.337141,"scheduling_latency_s":0.337163} gitlab # [ 258.091230] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.014Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"ab54cab88bf8a01db75a64a9","created_at":"2026-09-23T19:21:30.666Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.667Z","job_size_bytes":6,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-ab54cab88bf8a01db75a64a9: done: 0.010507 sec","job_status":"done","queue_duration_s":0.337141,"scheduling_latency_s":0.337163,"redis_calls":7,"redis_duration_s":0.0008370000000000001,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000607,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00023,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.006801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.010507,"completed_at":"2026-09-23T19:21:31.014Z","load_balancing_strategy":"primary","db_duration_s":0.000314,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.153183] gitaly[890]: time="2026-09-23T19:21:31.153Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:22:01.075" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:31.076" pid=890 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root gitlab # [ 258.153532] gitaly[890]: time="2026-09-23T19:21:31.153Z" level=info msg="optimized repository" clean_stale_data_time=13.846453ms clean_work_trees_time="70.4µs" commit_graph_time="3.632µs" correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="3.911µs" pid=890 prune_time="3.353µs" relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="4.191µs" storage_name=default system=housekeeping total_time=14.026643ms gitlab # [ 258.154028] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.015Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"6676e0c93dca7e0ed7181008","created_at":"2026-09-23T19:21:30.667Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.668Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-6676e0c93dca7e0ed7181008: start","job_status":"start","queue_duration_s":0.347528,"scheduling_latency_s":0.347541} gitlab # [ 258.159255] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.031Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"6676e0c93dca7e0ed7181008","created_at":"2026-09-23T19:21:30.667Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.668Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-6676e0c93dca7e0ed7181008: done: 0.015415 sec","job_status":"done","queue_duration_s":0.347528,"scheduling_latency_s":0.347541,"redis_calls":7,"redis_duration_s":0.000911,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000669,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000242,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015415,"completed_at":"2026-09-23T19:21:31.031Z","load_balancing_strategy":"primary","db_duration_s":0.002648,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.196879] gitaly[890]: time="2026-09-23T19:21:31.153Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-23T19:22:01.075" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-23T19:21:31.076" pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.205078] gitaly[890]: time="2026-09-23T19:21:31.182Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-23T19:22:01.181" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:31.181" grpc.time_ms=0.6 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.216729] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.032Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"a5c5bae1ed56668a6e575944","created_at":"2026-09-23T19:21:30.747Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.760Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a5c5bae1ed56668a6e575944: start","job_status":"start","queue_duration_s":0.271262,"scheduling_latency_s":0.271276} gitlab # [ 258.230747] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.206Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"a5c5bae1ed56668a6e575944","created_at":"2026-09-23T19:21:30.747Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.760Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":2530,"job_status":"rescheduled","message":"PipelineProcessWorker JID-a5c5bae1ed56668a6e575944: rescheduled"} gitlab # [ 258.270537] gitaly[890]: time="2026-09-23T19:21:31.200Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEHTFXKXFK0KBZE3NP5CY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-23T19:22:01.199" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-23T19:21:31.200" grpc.time_ms=0.498 pid=890 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 258.271256] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.210Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"a5c5bae1ed56668a6e575944","created_at":"2026-09-23T19:21:30.747Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-d3782ad506cb45a1","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:30.760Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a5c5bae1ed56668a6e575944: done: 0.178381 sec","job_status":"done","queue_duration_s":0.271262,"scheduling_latency_s":0.271276,"gitaly_calls":3,"gitaly_duration_s":0.073657,"redis_calls":68,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.009569,"redis_read_bytes":149,"redis_write_bytes":18168,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000328,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000791,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":22,"redis_queues_duration_s":0.003416,"redis_queues_read_bytes":24,"redis_queues_write_bytes":12310,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.002936,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000222,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000497,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001051,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079017,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.155099041,"duration_s":0.178381,"completed_at":"2026-09-23T19:21:31.210Z","load_balancing_strategy":"primary","db_duration_s":0.020967,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 258.337287] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.211Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"fa957803c1e2ee37bfd274f8","created_at":"2026-09-23T19:21:31.071Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.072Z","job_size_bytes":23,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-fa957803c1e2ee37bfd274f8: start","job_status":"start","queue_duration_s":0.138855,"scheduling_latency_s":0.138871} gitlab # [ 258.338076] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.235Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"fa957803c1e2ee37bfd274f8","created_at":"2026-09-23T19:21:31.071Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.072Z","job_size_bytes":23,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-fa957803c1e2ee37bfd274f8: done: 0.023538 sec","job_status":"done","queue_duration_s":0.138855,"scheduling_latency_s":0.138871,"redis_calls":7,"redis_duration_s":0.000907,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00067,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000237,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023538,"completed_at":"2026-09-23T19:21:31.235Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004259,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 258.352616] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.235Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a5493ede0fffdc4c3a274fd5","created_at":"2026-09-23T19:21:31.148Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.150Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a5493ede0fffdc4c3a274fd5: start","job_status":"start","queue_duration_s":0.08574,"scheduling_latency_s":0.085755} gitlab # [ 258.398209] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.250Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a5493ede0fffdc4c3a274fd5","created_at":"2026-09-23T19:21:31.148Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.150Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a5493ede0fffdc4c3a274fd5: done: 0.014778 sec","job_status":"done","queue_duration_s":0.08574,"scheduling_latency_s":0.085755,"redis_calls":7,"redis_duration_s":0.00088,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000625,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014778,"completed_at":"2026-09-23T19:21:31.250Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002237,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.404758] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.251Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f9dfcf5b0991897bb6949e6f","created_at":"2026-09-23T19:21:31.157Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.158Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f9dfcf5b0991897bb6949e6f: start","job_status":"start","queue_duration_s":0.092563,"scheduling_latency_s":0.092576} gitlab # [ 258.419095] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.266Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f9dfcf5b0991897bb6949e6f","created_at":"2026-09-23T19:21:31.157Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.158Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f9dfcf5b0991897bb6949e6f: done: 0.01529 sec","job_status":"done","queue_duration_s":0.092563,"scheduling_latency_s":0.092576,"redis_calls":7,"redis_duration_s":0.00088,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000651,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000229,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01529,"completed_at":"2026-09-23T19:21:31.266Z","load_balancing_strategy":"primary","db_duration_s":0.002446,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.461722] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.267Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"7cb520ff24975f7378f71798","created_at":"2026-09-23T19:21:31.164Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.166Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-7cb520ff24975f7378f71798: start","job_status":"start","queue_duration_s":0.101575,"scheduling_latency_s":0.101589} gitlab # [ 258.469701] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.284Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"7cb520ff24975f7378f71798","created_at":"2026-09-23T19:21:31.164Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.166Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-7cb520ff24975f7378f71798: done: 0.016707 sec","job_status":"done","queue_duration_s":0.101575,"scheduling_latency_s":0.101589,"redis_calls":10,"redis_duration_s":0.001265,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000618,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000361,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000286,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016707,"completed_at":"2026-09-23T19:21:31.284Z","load_balancing_strategy":"primary","db_duration_s":0.002517,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.511803] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.285Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"85924cc7cedc9456e5758d6c","created_at":"2026-09-23T19:21:31.166Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.167Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-85924cc7cedc9456e5758d6c: start","job_status":"start","queue_duration_s":0.117707,"scheduling_latency_s":0.117722} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses gitlab # [ 258.522446] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.299Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"85924cc7cedc9456e5758d6c","created_at":"2026-09-23T19:21:31.166Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.167Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-85924cc7cedc9456e5758d6c: done: 0.014121 sec","job_status":"done","queue_duration_s":0.117707,"scheduling_latency_s":0.117722,"redis_calls":7,"redis_duration_s":0.000937,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000236,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014121,"completed_at":"2026-09-23T19:21:31.299Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002191,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.572507] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.300Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"e0ffac1801c167b5bc21396a","created_at":"2026-09-23T19:21:31.167Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.169Z","job_size_bytes":19,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-e0ffac1801c167b5bc21396a: start","job_status":"start","queue_duration_s":0.13103,"scheduling_latency_s":0.131044} gitlab # [ 258.610254] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.313Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"e0ffac1801c167b5bc21396a","created_at":"2026-09-23T19:21:31.167Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.169Z","job_size_bytes":19,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-e0ffac1801c167b5bc21396a: done: 0.013219 sec","job_status":"done","queue_duration_s":0.13103,"scheduling_latency_s":0.131044,"redis_calls":8,"redis_duration_s":0.000987,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00023,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008685,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013219,"completed_at":"2026-09-23T19:21:31.313Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001954,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.611168] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.314Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ca49b2771d5856e8f71541e1","created_at":"2026-09-23T19:21:31.169Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.170Z","job_size_bytes":66,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ca49b2771d5856e8f71541e1: start","job_status":"start","queue_duration_s":0.143495,"scheduling_latency_s":0.14351} gitlab # [ 258.625845] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.326Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ca49b2771d5856e8f71541e1","created_at":"2026-09-23T19:21:31.169Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.170Z","job_size_bytes":66,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ca49b2771d5856e8f71541e1: done: 0.012313 sec","job_status":"done","queue_duration_s":0.143495,"scheduling_latency_s":0.14351,"redis_calls":7,"redis_duration_s":0.0009040000000000001,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000251,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012313,"completed_at":"2026-09-23T19:21:31.326Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001404,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.671488] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.327Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"2d61682aa27e7b2f1093fc72","created_at":"2026-09-23T19:21:31.171Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.171Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2d61682aa27e7b2f1093fc72: start","job_status":"start","queue_duration_s":0.15551,"scheduling_latency_s":0.155524} gitlab # [ 258.678755] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.342Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"Ci::PipelineFinishedWorker","jid":"2d61682aa27e7b2f1093fc72","created_at":"2026-09-23T19:21:31.171Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0a7c1b70389f5450c4d99f719cfa84d9d157db429a8cf22b1d3be2bf77e33077","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.171Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-2d61682aa27e7b2f1093fc72: done: 0.015075 sec","job_status":"done","queue_duration_s":0.15551,"scheduling_latency_s":0.155524,"redis_calls":7,"redis_duration_s":0.000858,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000613,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000245,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009202,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015075,"completed_at":"2026-09-23T19:21:31.342Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003564,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.725344] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.343Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"43d393ace8b8d147012d6c6e","created_at":"2026-09-23T19:21:31.172Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.173Z","job_size_bytes":6,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-43d393ace8b8d147012d6c6e: start","job_status":"start","queue_duration_s":0.170307,"scheduling_latency_s":0.170321} gitlab # [ 258.731027] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.351Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"43d393ace8b8d147012d6c6e","created_at":"2026-09-23T19:21:31.172Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.173Z","job_size_bytes":6,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-43d393ace8b8d147012d6c6e: done: 0.008087 sec","job_status":"done","queue_duration_s":0.170307,"scheduling_latency_s":0.170321,"redis_calls":7,"redis_duration_s":0.000928,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000622,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.005183,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.008087,"completed_at":"2026-09-23T19:21:31.351Z","load_balancing_strategy":"primary","db_duration_s":0.000231,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.768099] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.352Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"a9da8040cfedfbbeb98ca440","created_at":"2026-09-23T19:21:31.173Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.174Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a9da8040cfedfbbeb98ca440: start","job_status":"start","queue_duration_s":0.178161,"scheduling_latency_s":0.178174} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/263054a42f00ab5c2554e36527725bb48ab5e4c1/statuses, in 0.24 seconds) (finished: subtest: eval and build post commit statuses, in 10.52 seconds) (finished: run the VM test script, in 309.91 seconds) gitlab # [ 258.813506] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.365Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"a9da8040cfedfbbeb98ca440","created_at":"2026-09-23T19:21:31.173Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.174Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a9da8040cfedfbbeb98ca440: done: 0.012739 sec","job_status":"done","queue_duration_s":0.178161,"scheduling_latency_s":0.178174,"redis_calls":7,"redis_duration_s":0.000954,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000277,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012739,"completed_at":"2026-09-23T19:21:31.364Z","load_balancing_strategy":"primary","db_duration_s":0.001777,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 258.817951] sidekiq-cluster[2530]: {"severity":"INFO","time":"2026-09-23T19:21:31.365Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"fbf7d2c2bd3674dd879b8a01","created_at":"2026-09-23T19:21:31.206Z","trace_propagation_headers":{"sentry-trace":"0a35b58de2ef45c68dddb9b3b2fd53f7-b15e658326e24273","baggage":"sentry-trace_id=0a35b58de2ef45c68dddb9b3b2fd53f7,sentry-release=Unknown"},"correlation_id":"01M37VEHTFXKXFK0KBZE3NP5CY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-23T19:21:31.207Z","job_size_bytes":3,"pid":2530,"sidekiq_tid":"m7xy","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fbf7d2c2bd3674dd879b8a01: start","job_status":"start","queue_duration_s":0.15827,"scheduling_latency_s":0.158284} test script finished in 309.95s cleanup kill QemuMachine (pid 46) gitlab # [ 258.832747] gitaly[890]: time="2026-09-23T19:21:32.073Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M37VEKqemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/gxzhl7aaiid7zp3y47jqqiq7zg5mqpwp-python3-3.14.6/bin/python3.14) gitlab # M (finished: cleanup, in 1.53 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-23T19:20:34.844Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '263054a42f00ab5c2554e36527725bb48ab5e4c1', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-23T19:21:21.474610Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}] [] [{'id': 4, 'sha': '263054a42f00ab5c2554e36527725bb48ab5e4c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-23T19:21:21.687Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 4, 'sha': '263054a42f00ab5c2554e36527725bb48ab5e4c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-23T19:21:21.687Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 4, 'sha': '263054a42f00ab5c2554e36527725bb48ab5e4c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-23T19:21:21.687Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 4, 'sha': '263054a42f00ab5c2554e36527725bb48ab5e4c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-23T19:21:21.687Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 4, 'sha': '263054a42f00ab5c2554e36527725bb48ab5e4c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-23T19:21:21.687Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 4, 'sha': '263054a42f00ab5c2554e36527725bb48ab5e4c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-23T19:21:21.687Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 4, 'sha': '263054a42f00ab5c2554e36527725bb48ab5e4c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-23T19:21:21.687Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '263054a42f00ab5c2554e36527725bb48ab5e4c1', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'building attributes', 'created_at': '2026-09-23T19:21:30.545Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '263054a42f00ab5c2554e36527725bb48ab5e4c1', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 8s (1 warning)', 'created_at': '2026-09-23T19:21:30.689Z', 'started_at': None, 'finished_at': '2026-09-23T19:21:30.687Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '263054a42f00ab5c2554e36527725bb48ab5e4c1', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '1 attributes built', 'created_at': '2026-09-23T19:21:30.545Z', 'started_at': None, 'finished_at': '2026-09-23T19:21:30.802Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '263054a42f00ab5c2554e36527725bb48ab5e4c1', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 8s (1 warning)', 'created_at': '2026-09-23T19:21:30.689Z', 'started_at': None, 'finished_at': '2026-09-23T19:21:30.687Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]